5. 3 Conclus ion
Chapter 6 Testing
6.3 Performance Testing
6.3.4 Se<'llTIt (.im e
To te"t th€ seflrch time we had considered a number of I}{,Imulfltions of ill(' numfx:1' of files flIld the number of keywords l}{']' tile, for em'll l:>ermlltiltioll, a llUBI!>"r of It.:b lil~,
were Ilploadpd with a1ll1ml",r of kp):words "tbd",d. Thp keywords u,."d W€re tak€n from a
1,.\ Slntic field b a field ... 1><>6< d"",n', change ",hen Uw input i, dlanged or d"", not add to the ""cmil}, of the SFO, ,neh "" File "run"..
University
of Cape
Town
100 90 80 70 60
"
4030
)0
10
o
Percentage Overhead
+ + - - ---
- --- loc
c
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~
"
~"
~ ~"
~" "
~ ~ ~~ N ~ 0 C N
"
~ 0 0"
N M ~ N ~,
~ 00 M 0 M M ~ 0 ~Fignn' 6.1' T)w physic"l '1,0['4;" ""!'rhead ''''1'n",<"('d a~ a l'ercelll.ag€ of the input file ",il~_ It it< shown that as the fil~ siz€~ ar€ increa~ed then the on'rhead becomes negligIble.
dictionary thell stored in a list to be Ilsed for th" query. Ouce Ilploaded th" syslelll KClllid then IauriOll1jy sc[,,("t a k,,:,,'w()rd [rom th" otOIcd lisl and "u1>m;l, il. to I.h" EC2 i;('[\·ice. The EC2 ",[vie" I.hc" performed" scan,h I.hnmgh all dH' file" in the S3 h\l~kel, rNurning the
mat~hes. figure G.J ",hOWl; an owrview, at an architectural level, of the tests perfomml.
Cli""i l'erf()rmimCC
I" l·esting the dient performance. we upJr,aded a llulllb<,r of files each with a stat;" number of keywords al,l.ad,,,d alld d,e]) t("'<ted the re;pOll5e time, this ww repeated 30 times. !'uch a 1.""1, giY{'~ all illdical,ioll of Lhe Ilsahility impact. Users will not be willing to wait lOllg
University
of Cape
Town
'OC{IJ
"'"
, " i
""
c
,
0S
~JO[IJ1
2OC{IJ .,=
0 Re1l:b
Secure Upload Tillle 'IS Unsecure
U~oadTime
fl~,Fit
~
..
-;:iI!1Cllb File Si;oes
periods of time for a response from the server,
" Ayg5e(nCiF.fi:
Avg c·,>eel" cc:n
"A'iq I, 1xW! SJ
-Av~5e(u'!S<
The resn]ts of this I.:SI, are shown in Figure (j.(j, The !-\taph shows how til(' wspom;<: Ii"",
i"cr~as,'" as the llumber 01' files iu an S3 bllCkN i!lrr~ases. The r~a.'>On for the drastic increaoe in response time between forty tiles alld one hnndrro files is dne to n('[.work lal."uci,>; wi,hill Amazoll's lIlh·'~'l.nj('I.11l'<' wll('u dowuloadillg "11" IlllIldf('d k""word" fil~ trom 83 to Ee2 as opposed 1:0 ollly downloading fony tile8_
Server Performance
Although tesl.mg Ihe di,'llI. p"rformaJ](" is Il(',>ded I.() ""amill" how I()ug Hs"rs ,,,,'I,d 1:0 wai, [or T"SPOIlSl:S, i, d')("Il' I. acrHral,,'lv show til(' ]H'l'torItlHlKe of ,h~ oeal'rh algorithm. Ab the Illlllll",r of k"yword, incl'ea""," the ro:><pom;e time remain8 COll8tant thi:, b be(:ause the di[·
[,'[('ur"s af(' dislorkd by lav'uc)' rhallges. To accura(ely tl";t dw performance degradation
University
of Cape
Town
Cllen! Side Con'pute Cloud IE(1)
Scnd"O ~ K",,.~d
'0
c""'pu'o <Iou"v,.ycr'o,'"
(C" ' ~ ;,",''' O'')- ~
, '"".e,',,_
r
,.,".,,, -~.y'''>'d.
to
"o.
O<J' 50"""Sto'''g'' Cloud (S3)
Figur~ 6_5: Th~ test en"ironm~nt overview for the t€,ting performed "t the client and th€ l'erwr, There needs to he two levels of testing, fwm tbe di"nt i'ide ,md from the sen-€l' i'ide. The dient I'ide ind",jes lal.{~lCy i,Stl(S m,d exalHiI"'~ bow IOllg a cil('1I1. Im<, to wail. foT' a ~('arcb query re>po11'e_
')'be s<'r\'('r oi(1<, (,X"JHlIl('S IIL€ perfoT'lllallcr of I.he actual S€arch quer), eliminating latency i"ue,
as I.he number of ke),Y,rord> increaS€> we duwnlow:ied hilihe files from 83 llltO hll army in lhe ECl instan('e's memory, thus removing b,l en('y ii'suei' a.nd red udng the nnmber of p'l,<I;e fanlt" thai ('(mid (K'(,ur by 11~ing ol,her d".1,,, ~l.r11<"1,llT'('~ Oll('e 1.11" til€> w~r~ ill memory the applinll.ion il,,,nl.l,,,d o\-!'[ Ih" tik~ "","'11I,illg Ihe "'arel, fnllcl.ion on each it~m,
' re initiall,' stn.rte(llhe testing by llsing" di(,tionary to sded l.lw nlIldOllL keyword>
I hill were al.l,,,.:::he<i to th€ Secnre File Obj~cb_ TILe raIldom hywords w€re then mved to "
tile i'O that we could use them li1t~r for the the seim'h, The tesl, would randomly ,,"]()('I, a
fil~ from the "USN keywords" file hnd mbmit I.his 1,0 the EC2 insl aJl('(" I hi~ """,, T'epeal,,,j Iwenty Limes. The Sen-cr, nllming on I,he E('2 iIlst,iUlC", would then mea'ure OO\\' long it lo.,k to il,('mtc ,'\U "aclL keywoT'd withiu eaclL fil~_ Thes€ firsl te,ts were timed .Li'ing milli"", ",d pf()('isioH m,d I h" [""nil" of I,h",,~ te'ts are shown in Figure G. 7,
University
of Cape
Town
~
~ c
0 v
> •
40000 35000 30000
25000 20000
l
15000 ..t 10000
SCXlO
l L .- ..
0 '
...
-5 Keywoms 10 Key,<"ums 100 Keywords
1 Keyword 7 Ke'f'/l'wls 50 Ke}'\wrds
- lFlle -lOFles
20 F!~5
-401-';"5 - 100 FI'es
Fi";l1H' 6,G: 'I'll(' a"enl!'," lim" l,aJ"," for a di(,IlI 10 gd resl1lt~ hack from the "",rver applkation_ Tlw graph ~how~ the tim~ takpn for varying numbers of file:, stored in lin S3 hl1ckct and varying numbers of hvwords.
Thf problem wit h doing t\\'eIlty Iterations of fach test ca:oe Wab that \\'e \wre getting some unexplainablt, result.s when cDmparing the actual rt'sults with l,l](' I.lworeli('al allalvsis of the al,.;oridull, \\'(' 1I",n dedded 1,0 l,rv llaIH"""'OIld pr('c:isioll Whfll doiIlg the twenty it-eratiOllS of the t~st cas~s, sinc~ some of tlw re,ults we were g(,tting were in sub-millisecond pt'rformllnce, However as shown in Figure (j,8 there wef(' still unexplainabl(' n,,,,its <XJmiIlg from Ill('se leSIS. 'I'll(' moSl, obvi,ms a11O",,,I:-' is when, d](' perf{)rlllaIlC~ g~ts bettfr from 100 files with ,50 k~yword~ to 100 fil~s with 100 kpywords. Ohviously !j000 compare opt'fations slJ(mid lake less tim~ to complpte than 100011 For this reabOll \\'f decidE'C1 to perform 100 iterations of each te~t cab<', hoping to get le:;~ lloisy data. Thl' results of 1 his are showIL in
University
of Cape
Town
, •
•
... ..
.. ..
.. .. ,
... .. ,
--=
-,
~..
,- '" ,<,
~ <.~...
•- ..
~ ~.~,-,,,,,~,~
..
Figun.' 6i Thi,; hgur .. :;ho"'~ lhp p,·rf(.rmflnce de~mdaLi"n uf ,.,,,yin;:. tIl<' Il'"uh~r
of kevuurrn
ror II rLXe-d
nUiuber( . r
nk". when,",,1\-
10 it(>ratiQn~ \n;~J"f'dr,"e.
Figllr~ 6,9 and a~ can be se~n the~'I\le fl.id~· diITer~lll 10 Ih" I.w"my ",'tst. .. :>n r~'\llt~. An
otlwr chnuge I,hal. waS ffiHd" ""~, I
c .
h ,kf",.
h ,'''''ord' that. w~rp not u'<ed wlwn C!"~(li ing Illl' w"t 1".\ Tili, W!l1lld tor{"p rlo .. algonthm 10 perinl'm cnmlJare "JwmLlmt._ .. ,, .,)1 h,.word, ill 311 hl~ a.~ 01'PObed 10 se'lIclJinp. Ihr"lIl':h tht' k''''"urols "f a fil,- 1mdl II mllleh i~ t'oundthll~ lhting L1,e wor~1 ,."so·
Itcferring hack to th .. u.e~ ot"<'t> ";:.>11 II. It tiluoi,- ;""r\'i(~, nHJld storp "'"~1( hie; in folders 1,;\-00 Oll the art"t SCI il ,,,,,8 bi.ll .. ,.,1,'1<> .~ .... ",up that you cDulrl j!:et foid<.'L'li "lIh ((I ni,>;
or even up i{) JIM) files, We 'nil\, m,'IUW' that "" a. ,><'r'0lll1.1 computeI', in geur!0111l;i:'f" will n,,[ lww lJlon' t.hall 1
University
()(l file' in t·h.·;" folrlo'r"of Cape
Town
, , •
•
1 •
•
•
" ,
," ..
- ,
•
•
----•
""-,,,.
• , "" . .
'~Figllre 6.8: Thi, iiglmc shows the p€rformatlC€ degradat.ion when doing 20 i1(,nl-t.ions of each 11'" cas" wiLl! Il<Ul()s<'Coud precisioll, note the anomaly happening from 100 files with 50 k{·.\'·w()]d~ 10 lOll iile~ with 1011 ke,I'-words
s". ,,,"
r .. ,~.-•• -
-
0'n ... " .".-
0 ._.
-
~'N'-
-Fi[:';urc (i.9: Thi~ fi~Ul'e show~ the per[ormrulc;c degradation of varying tlw !lumber of keywords for a fixed llllmllel' of files, where the test eases w~re rep.catro
lOll Liaws with llill.j(~Jlld precision ru, "-ell as tebt.ing the worst C8.."€
e&h time.