Professional Documents
Culture Documents
TABLE OF CONTENTS
1 2 Introduction Requirements 2.1 Java Version 2.2 Operating Systems 3 Features 4 Running JMeter !ui"ding a test p"an .1 #dding and removing e"ements .2 $oading and saving e"ements .3 %on&iguring tree e"ements .4 Running a test p"an . Scoping Ru"es ' ("ements o& a test p"an '.1 )*read +roup '.2 %ontro""ers '.2.1 Samp"ers '.2.2 $ogic contro""ers '.3 $isteners '.4 )imers '. #ssertions '.' %on&iguration e"ements '., -re.-rocessor e"ements './ -ost.-rocessor e"ements '.0 (1ecution Order , !ui"ding a 2e3 test p"an ,.1 #dding 4sers ,.2 #dding de&au"t 5))- request properties ,.3 #dding coo6ie support ,.4 #dding 5))- requests ,. #dding a "istener to vie2 store t*e test resu"ts ,.' Saving t*e test p"an ,., Running t*e test p"an / !ui"ding an advanced 2e3 test p"an /.1 5and"ing user sessions 2it* 4R$ re2riting /.1.1 4R$ Re2riting (1amp"e /.2 4sing a *eader manager 0 $isteners 0.1 Screen captures 0.2 Resource usage 0.3 Saving response data 17 !est -ractices 17.1 $imit t*e num3er o& t*reads 17.2 4ser varia3"es 11 Functions 11.1 %*aracteristics o& &unction 11.2 Function8s usage 11.3 9riting t*e &unction string 11.4 )*e &unction *e"per dia"og
12 13
1.
Introduction
#pac*e JMeter is a 100% pure Java desktop application designed to load test client/server software (such as a web application). It ma be used to test performance both on static and d namic resources such as static files! Java "ervlets! #$I scripts! Java ob%ects! databases! &'( servers and more. J)eter can be used to simulate a heav load on a server! network or ob%ect to test its strength or to anal *e overall performance under different load t pes. +dditionall ! J)eter can help in regression test the application b letting us create the test scripts with assertions to validate that the application is returning the results we e,pect. &or ma,imum fle,ibilit ! J)eter lets us create these assertions using regular e,pressions.
2.
2.1
Requirements
Java Version
J)eter re-uires a full compliant J.) 1./ or higher. It is e,pected J)eter performs best with 1.0 or better.
2.2
Operating Systems
J)eter run correctl on an s stem that has a compliant Java implementation. J)eter has been tested and works under1 I. 2ni, ("olaris! 3inu,! etc) II. 4indows (56! 7'! 8000! 9() III. :pen.)" +lpha ;./<
3.
Features
I.
#an load and performance test =''( and &'( servers as well as arbitrar database -ueries (via J>?#) II. #omplete portabilit and 177: Java purity. III. &ull S2ing and lightweight component support I.. &ull mu"tit*reading framework allows concurrent sampling b man threads and simultaneous sampling of different functions b separate thread groups. .. #areful +4I design allows faster operation and more precise timings. .I. #aching and offline anal sis/repla ing of test results. .II. =ighl @,tensible1 +. (luggable "amplers allow unlimited testing capabilities. ?. "everal load statistics ma be chosen with p"ugga3"e timers. #. >ata anal sis and visua"i;ation p"ugging allow great e,tendibilit as well as personali*ation. >. &unctions (which include Java"cript) can be used to provide d namic input to a test @. "criptable "amplers (?ean "hell is supported in version 1.5.8 and above)
4.
Running JMeter
'o run J)eter! run the %meter.bat (for 4indows) or %meter (for 27I9) file. J)eter must be started from the J)eter bin director (where those files are found). 'he %meter.bat file attempts to change to the appropriate director if it can.
+ test plan describes a series of steps J)eter will e,ecute when run. + complete test plan will consist of one or more 'hread $roups! logic controllers! sample generating controllers! listeners! timers! assertions! and configuration elements.
.1
+dding elements to a test plan can be done b rightAclicking on an element in the tree! and choosing a new element from the BaddB list. +lternativel ! elements can be loaded from file and added b choosing the BopenB option. 'o remove an element! make sure the element is selected! rightAclick on the element! and choose the BremoveB option.
Scoping Ru"es
'he J)eter test tree contains elements that are both hierarchical and ordered. "ome elements in the test trees are strictl hierarchical (3isteners! #onfig @lements! (ostA(rocessors! (reA (rocessors! +ssertions! 'imers)! and some are primaril ordered (controllers! samplers). 4hen we create our test plan! ou will create an ordered list of sample re-uest (via "amplers) that represent a set of steps to be e,ecuted. 'hese re-uests are often organi*ed within controllers that are also ordered. $iven the following test tree1
'he order of re-uests will be! :ne! 'wo! 'hree! &our. "ome controllers affect the order of their sub elements! and controllers in the component reference. ou can read about these specific
:ther elements are hierarchical. +n +ssertion! for instance! is hierarchical in the test tree. If its parent is a re-uest! then it is applied to that re-uest. If its parent is a #ontroller! then it affects all re-uests that are descendants of that #ontroller. In the following test tree1
+ssertion D1 is applied onl to Ee-uest :ne! while +ssertion D8 is applied to Ee-uests 'wo and 'hree. +nother e,ample! this time using 'imers1
In this e,ample! the re-uests are named to reflect the order in which the will be e,ecuted. 'imer D1 will appl to Ee-uests 'wo! 'hree! and &our (notice how order is irrelevant for
onl
'.
'he 'est (lan ob%ect has a new checkbo, to select called B&unctional 'estingB. If selected! it will cause J)eter to record the data returned from the server for each sample. If we have selected a file in our test listeners! this data will be written to file. 'his can be useful if we are doing a small run to ensure that J)eter is configured correctl ! and that our server is returning the e,pected results. 'he conse-uence is that the file will grow huge -uickl ! and J)eterCs performance will suffer. 'his option should be off if we are doing stressAtesting (it is off b default). If we are not recording the data to file! this option makes no difference.
'.2 %ontro""ers
J)eter has two t pes of #ontrollers1 "amplers and 3ogical #ontrollers. "amplers tell J)eter to send re-uests to a server. &or e,ample! add an =''( Ee-uest "ampler if we want J)eter to send an =''( re-uest. 4e can also customi*e a re-uest b adding one or more #onfiguration @lements to a "ampler.
3ogical #ontrollers let us customi*e the logic that J)eter uses to decide when to send re-uests. &or e,ample! we can add an Interleave 3ogic #ontroller to alternate between two =''( Ee-uest "amplers.
'.2.1 Samp"ers
"amplers tell J)eter to send re-uests to a server. J)eter samplers include1 I. &'( Ee-uest II. =''( Ee-uest III. J>?# Ee-uest I.. Java ob%ect re-uest .. 3>+( Ee-uest .I. ":+(/9)3AE(# Ee-uest .II. 4eb "ervice (":+() Ee-uest @ach sampler has several properties we can set. 4e can further customi*e a sampler b adding one or more #onfiguration @lements to it. <ote= J)eter sends re-uests in the order that the samplers appear in the tree. If we are sending multiple re-uests of the same t pe (for e,ample! =''( Ee-uest) to the same server! consider using a >efaults #onfiguration @lement. @ach controller has one or more >efaults elements. +lso! add a 3istener to our 'hread $roup to view and/or store the results of our re-uests.
'he first thing about this test is that the login re-uest will be e,ecuted onl the first time through. "ubse-uent iterations will skip it. 'his is due to the effects of the :nce :nl #ontroller.
+fter the login! the ne,t "ampler loads the search page (imagine a web application where the user logs in! and then goes to a search page to do a search). 'his is %ust a simple re-uest! not filtered through an 3ogic #ontroller. +fter loading the search page! we want to do a search. +ctuall ! we want to do two different searches. =owever! we want to reAload the search page itself between each search. 4e could do this b having 0 simple =''( re-uest elements (load search! search B+B! load search! search B?B). Instead! we use the Interleave #ontroller which passes on one child re-uest each time through the test. It keeps the ordering (i.e. A it doesnCt pass one on at random! but BremembersB its place) of its child elements. Interleaving 8 child re-uests ma be overkill! but there could easil have been 6! or 80 child re-uests. <ote= 'he =''( Ee-uest >efaults that belongs to the Interleave #ontroller. Imagine that B"earch +B and B"earch ?B share the same (+'= info. 'his makes sense A both are search re-uests! hitting the same backAend search engine. Eather than configure both =''( "amplers with the same information in their (+'= field! we can abstract that information out to a single #onfiguration @lement. 4hen the Interleave #ontroller Bpasses onB re-uests from B"earch +B or B"earch ?B! it will fill in the blanks with values from the =''( default re-uest #onfiguration @lement. "o! we leave the (+'= field blank for those re-uests! and put that information into the #onfiguration @lement. In this case! this is a minor benefit at best! but it demonstrates the feature. 'he ne,t element in the tree is another =''( default re-uest! this time added to the 'hread $roup itself. 'he 'hread $roup has a builtAin 3ogic #ontroller! and thus! it uses this #onfiguration @lement e,actl as described above. It fills in the blanks of an Ee-uest that passes through. It is e,tremel useful in web testing to leave the >:)+I7 field blank in all our =''( "ampler elements! and instead! put that information into an =''( default re-uest element! added to the 'hread $roup. ? doing so! we can test our application on a different server simpl b changing one field in our 'est (lan. :therwise! we have to edit each and ever "ampler. 'he last element is a =''( #ookie )anager. + #ookie )anager should be added to all web tests A otherwise J)eter will ignore cookies. ? adding it at the 'hread $roup level! we ensure that all =''( re-uests will share the same cookies. 3ogic #ontrollers can be combined to achieve various results.
'.3 $isteners
3isteners provide access to the information J)eter gathers about the test cases while J)eter runs. 'he graph result listener plots the response times on a graph. 'he B.iew Eesults 'reeB 3istener shows details of sampler re-uests and responses! and can displa basic =')3 and 9)3 representations of the response. :ther listeners provide summar or aggregation information. +dditionall ! listeners can direct the data the collect to a file for later use. @ver J)eter provides a field to indicate the file to store data to. 3isteners can be added an where in the test. 'he below their level. will collect data onl listener in
from elements at or
'.4 )imers
10
? default! a J)eter thread sends re-uests without pausing between each re-uest. 4e can specif a dela b adding one of the available timers to our 'hread $roup. If we do not add a dela ! J)eter could overwhelm our server b making too man re-uests in a ver short amount of time. 'he timer will cause J)eter to dela thread makes. a certain amount of time between each re-uest that a
If we choose to add more than one timer to a 'hread $roup! J)eter takes the sum of the timers and pauses for that amount of time before e,ecuting the samplers to which the appl .
'.
#ssertions
+ssertions allow us to assert facts about responses received from the server being tested. 2sing an assertion! we can essentiall BtestB that our application is returning the results we e,pect it to. 4e can add an assertion to an "ampler 'o view the assertion results! we can add an +ssertion 3istener to the 'hread $roup.
11
+ (reA(rocessor e,ecutes some action prior to a "ampler Ee-uest being made. If a (reA (rocessor is attached to a "ampler element! then it will e,ecute %ust prior to that sampler element running. + (reA(rocessor is most often used to modif the settings of a "ample Ee-uest %ust before it runs! or to update variables that arenCt e,tracted from response te,t.
,.
In this section describes how to create a basic test plan to test a 4eb site. 4e will create five users that send re-uests to two pages on the Jakarta 4eb site. +lso! we will tell the users to run their tests twice. "o! the total number of re-uests is (G users) , (8 re-uests) , (repeat 8 times) H 80 =''( re-uests. 'o construct the 'est (lan! we will use the following elements1 'hread $roup! =''( Ee-uest! =''( Ee-uest >efaults! and $raph Eesults.
4e should now see the 'hread $roup element under 'est (lan. If we do not see the element! then Be,pandB the 'est (lan tree b clicking on the 'est (lan element. 7e,t! we need to modif the default properties. "elect the 'hread $roup element in the tree! if we have not alread selected it. 4e should now see the 'hread $roup #ontrol (anel in the right section of the J)eter window.
12
"tart b providing a more descriptive name for our 'hread $roup. In the name field! enter Jakarta 2sers. 7e,t! increase the number of users (called threads) to G. In the ne,t field! the EampA2p (eriod! leave the the default value of 0 seconds. 'his propert tells J)eter how long to dela between starting each user. &or e,ample! if we enter a EampA2p (eriod of G seconds! J)eter will finish starting all of our users b the end of the G seconds. "o! if we have G users and a G second EampA2p (eriod! then the dela between starting users would be 1 second (G users / G seconds H 1 user per second). If we set the value to 0! then J)eter will immediatel start all of our users. &inall ! clear the checkbo, labeled B&oreverB! and enter a value of 8 in the 3oop #ount field. 'his propert tells J)eter how man times to repeat our test. If we enter a loop count value of 1! then J)eter will run our test onl once. 'o have J)eter repeatedl run our 'est (lan! select the &orever checkbo,. 'he completed Jakarta 2sers 'hread $roup.
13
<otes= In most applications! ou have to manuall accept changes ou make in a #ontrol (anel. =owever! in J)eter! the #ontrol (anel automaticall accepts our changes as ou make them. If ou change the name of an element! the tree will be updated with the new te,t after ou leave the #ontrol (anel (for e,ample! when selecting another tree element).
?egin b selecting the Jakarta 2sers element. Eight click mouse AAI +dd menu AAI +dd AAI #onfig @lement AAI =''( Ee-uest >efaults. 'hen! select this new element to view its #ontrol (anel.
3ike most J)eter elements! the =''( re-uest defaults #ontrol (anel has a name field that we can modif . In this e,ample! leave this field with the default value. "kip to the ne,t field! which is the 4eb "erverCs "erver 7ame/I(. &or the 'est (lan that we are building! all =''( re-uests will be sent to the same 4eb server! %akarta.apache.org. @nter this domain name into the field. 'his is the onl field that we will specif a default! so leave the remaining fields with their default values. <otes= 'he =''( Ee-uest >efaults element does not tell J)eter to send an =''( re-uest. It simpl defines the default values that the =''( Ee-uest elements use. 'he completed =''( Ee-uest >efaults element
14
15
7e,t! add the second =''( Ee-uest and edit the following properties I. #hange the 7ame field to B(ro%ect $uidelinesB. II. "et the (ath field to B/site/guidelines.htmlB.
16
,.
'he final element we need to add to our 'est (lan is a 3istener. 'his element is responsible for storing all of the results of our =''( re-uests in a file and presenting a visual model of the data. "elect the Jakarta 2sers element and add a graph results listener (+dd AAI 3istener AAI $raph Eesults). 7e,t! we need to specif a director and filename of the output file. 4e can either t pe it into the filename field! or select the ?rowse button and browse to a director and then enter a filename.
17
/.
18
In ne,t &igure! we see the 2E3 EeAwriting modifier $2I! which %ust has a field for the user to specif the name of the session I> parameter. 'here is also a checkbo, for indicating that the session I> should be part of the path (separated b a BFB)! rather than a re-uest parameter
0.
$isteners
+ listener is a component that shows the results of the samples. 'he results can be shown in a tree! tables! and graphs or simpl written to a log file. 'o view the contents of a response from an given sampler! add either of the 3isteners B.iew Eesults 'reeB or B.iew Eesults in tableB to a test plan. 'o view the response time graphicall ! add graph results! spline results or distribution graph. 'he listeners section of the components page has full descriptions of all the listeners. 'he B#onfigureB button can be used to specif which fields to write to the file! and whether to write it as #". or 9)3. #". files are much smaller than 9)3 files! so use #". if ou are generating lots of samples.
19
If ou onl wish to record certain samples! add the 3istener as a child of the sampler. :r ou can use a "imple #ontroller to group a set of samplers! and add the 3istener to that. 'he same filename can be used b multiple samplers A but make sure the all use the same configurationK <ote= >ifferent listeners displa the response information in different wa s. =owever! the the same raw data to the output file A if one is specified. all write
20
:ur hardwareCs capabilities will limit the number of threads we can effectivel run with J)eter. It will also depend on how fast our server is (a faster server gives makes J)eter work harder since it returns re-uest -uicker). 'he more J)eter works! the less accurate its timing information will be. 'he more work J)eter does! the more each thread has to wait to get access to the #(2! the more inflated the timing information gets. If we need largeAscale load testing! consider running multiple nonA$2I J)eter instances on multiple machines.
17.2
4ser varia3"es
"ome test plans need to use different values for different users/threads. &or e,ample! we might want to test a se-uence that re-uires a uni-ue login for each user. 'his is eas to achieve with the facilities provided b J)eter. &or e,ample1
#reate a te,t file containing the user names and passwords! separated b commas. (ut this in the same director as our test plan. +dd a #". >ataset configuration element to the test plan. 7ame the variables 2"@E and (+"". Eeplace the login name with LM2"@EN and the password with LM(+""N on the appropriate samplers
'he #". >ata "et element will read a new line for each thread.
11. Functions
J)eter functions are special values that can populate fields of an configuration element in a test tree. + function call looks like this1 LMOOfunction7ame(var1!var8!var/)N 4here BOOfunction7ameB matches the name of a function. "ampler or other
(arentheses surround the parameters sent to the function! for e,ample LMOOtime(P)>)N 'he actual parameters var from function to function. &unctions that re-uire no parameters can leave off the parentheses! for e,ample LMOOthread7umN.
11.1
'here are two kinds of functions1 userAdefined static values (or variables)! and builtAin functions. 2serAdefined static values allow the user to define variables to be replaced with their static value when a test tree is compiled and submitted to be run. 'his replacement happens once at the beginning of the test run. 'his could be used to replace the >:)+I7 field of all =''( re-uests! for e,ample A making it a simple matter to change a test to target a different server with the same test. 'his t pe of replacement is possible without functions! but was less convenient and less intuitive. It re-uired users to create default config elements that would fill in blank values of "amplers. 2serAdefined functions allow one to replace onl part of an given value! not %ust filling in blank values.
21
4ith builtAin functions users can compute new values at runAtime based on previous response data! which thread the function is in! the time! and man other sources. 'hese values are generated fresh for ever re-uest throughout the course of the test. <ote= &unctions are shared between threads. @ach occurrence of a function call in a test plan is handled b a separate function instance.
11.2
Function8s usage
+ userAdefined function can be written into an field of an test component. "ome fields do not allow random strings because the are e,pecting numbers! and thus will not accept a function. =owever! most fields will allow functions. ?uiltAin functions can be written into an field of nonAcontroller test components. 'his includes "amplers! 'imers! 3isteners! )odifiers! +ssertions! (reA(rocessors! (ostA(rocessors and #onfig @lements.
11.3
2serAdefined functions take the form1 LMvar7ameN . In the 'est(lan tree element! a twoA column table of userAdefined values is kept! matching up variable names with static values. Eeferencing the variable in a test element is done b bracketing the variable name with CLMC and CNC. ?uiltAin functions are written in the same manner! but b convention! the names of builtAin parameters begin with BOOB to avoid conflict with user value names Q . "ome functions take arguments to configure them! and these go in parentheses! commaAdelimited. If the function takes no arguments! the parentheses can be left out. + further complication for argument values that themselves contain commas is that the value should be escaped as necessar . 'hus! if we need to include a comma in our parameter value! escape it like so1 CR!C. J)eter provides a tool to help us construct function calls for various builtAin functions! which we can then cop A paste. It will not automaticall escape values for us! since functions can be parameters to other functions! and we should onl escape values we intend as literal.
11.4
22
2sing the &unction =elper! we can select a function from the pull down! and assign values for its arguments. 'he left column in the table provides a brief description of the argument! and the right column is where we write in the value for that argument. >ifferent functions take different arguments. :nce we have done this! click the BgenerateB button! and the appropriate string is generated for us to cop Apaste into our test plan wherever we like.
23
'he ne,t figure shows how these shared variables are declared in J)eter. &or eg. .alue for &irst 7ame here is 'esting. 3ikewise we can define an value for the shared variables.
24
13. Re&erence
I.
%akarta.apache.org
25