Author :Vikas Singh

What is Performance Testing???

Performance testing is defined as the technical

investigation done to determine or validate the speed, scalability, and/or stability characteristics of the product under test. Performance-related activities, such as testing and tuning, are concerned with achieving response times, throughput, and resourceutilization levels that meet the performance objectives for the application under test.

Key Terms of Performance Testing

Performance test :To determine or validate speed, scalability, and/or stability. Load test :To verify application behavior under normal and peak load conditions. Stress test : To determine or validate an applications behavior when it is pushed beyond normal or peak load conditions. Baselines :Creating a baseline is the process of running a set of tests to capture performance metric data for the purpose of evaluating the effectiveness of subsequent performance-improving changes to the system or application. Benchmarking :Benchmarking is the process of comparing your systems performance against a baseline that you have created internally or against an industry standard endorsed by some other organization.

Why Performance Test?

Speed - Does the application respond quickly enough

for the intended users? Scalability Will the application handle the expected user load and beyond? Stability Is the application stable under expected and unexpected user loads? Confidence Are you sure that users will have a positive experience on go-live day?

Speed
User Expectations Experience Psychology Usage System Constraints Hardware Network Software

Scalability
How many users

before it gets slow? before it stops working? will it sustain? do I expect today? do I expect before the next upgrade?

How much data can it hold?

Database capacity File Server capacity Back-up Server capacity Data growth rates

Stability
What happens if there are more users than we expect? all the users do the same thing? a user gets disconnected? there is a Denial of Service Attack? the web server goes down? we get too many orders for the same thing?

Confidence
If you know what the performance is
you can assess risk. you can make informed decisions.

you can plan for the future.

you can sleep the night before go-live day.

The peace of mind that it will work on go-live day alone justifies the cost of performance testing.

Core Performance-Testing Activities

The seven core performance-testing activities can be summarized as follows:

Performance Testing Tools

Open Source Tools

JMeter http_load The Grinder Siege Commercial Tools LoadRunner (around $200,000 to own!) WebLoad Third-party $ervices

Why We Chose JMeter for Internal Testing

Easy to install and use

Free! Java most platforms Just download and run!

Feature-rich

Post forms Record from browser Load test data from files Add logic, variables & functions Run one test from multiple machines Test many protocols, not just HTTP e.g .Http , Https , Ftp , Smtp

JMeter's Nomenclature
User

-> Request -> Report ->

Thread Sampler Listener

Test Plan
The Test Plan is where the overall settings for a test are specified.

Static variables can be defined for values that are repeated throughout a test, such as server

names. And can be accessed by the $(server) variable inside the test plan.
Selecting Functional Testing instructs JMeter to save the additional sample information Response Data and Sampler Data - to all result files. This increases the resources needed to run a test, and may adversely impact JMeter performance.

Thread Group
A Thread Group defines a pool of users that will execute a particular test case against your server. In the Thread Group GUI, you can control the number of users simulated (num of threads), the ramp up time (how long it takes to start all the threads), the number of times to perform the test, and optionally, a start and stop time for the test.

Configuration Element

Sampler
CSV Data Set Config:
CSV Data Set Config is used to read lines from a file, and split them into variables. By default, the file is only opened once, and each thread will use a different line from the file. Lines are read at the start of each test iteration. The file name and mode are resolved in the first iteration.

HTTP Cookie Manager:

It stores and sends cookies just like a web browser. If there is an HTTP Request and the response contains a cookie, the Cookie Manager automatically stores that cookie and will use it for all future requests to that particular web site. Each JMeter thread has its own "cookie storage area". So, if you are testing a web site that uses a cookie for storing session information, each JMeter thread will have its own session. Received Cookies are stored as
JMeter thread variable

Cookie stored in the Cookie Manager: This gives you the opportunity to use hardcoded cookies that will be used by all threads during the test execution. Not Recommended.

HTTP Request Default

Result of HTTP Request Default

User Defined Variable

Result of UDV

FTP Request Default

JDBC Connection Configuration

Controllers

Simple Controller
The Simple Logic Controller lets you organize your Samplers and other Logic Controllers. Unlike other Logic Controllers, this controller provides no functionality beyond that of a
storage device

Simple Controller result

Loop Controller
Loop Controller in JMeter will loop through the request defined under it certain number of times, in addition to the loop value specified for the Thread Group.

Results of Loop Count Controller

It will iterate through the samplers as given in Loop Count.

Once only Controller

The Once Only Logic Controller tells JMeter to process the controller(s) inside it only once,and pass over any requests under it during further iterations through the test plan. If it is defined under loop controller having Loop Count = 3, it will execute the sampler under once only controller once.

Once Only Controller

Results of Once only Controller: Loop Count under

Loop controller is defined to be 3. But once only controller executes sampler under it once. But Once only controller have no effect on Loop count defined under thread Group.

Logic Controller

Include Controller
The include controller is designed to use an external jmx file.

To use it, add samples to a simple controller, then save the simple

controller as a jmx file.

Logic Controller
Include Controller contd

Runtime Controller
The Runtime Controller controls how long its children are allowed to run.

If Controller
The If Controller allows the user to control whether the test elements below it (its children) are run or not. Condition: Conditions are given to make sure that all the samplers below if controller are executed only when condition is met. Evaluate for all children: Evaluate for all children, is given to evaluate condition for all children. Otherwise it is evaluated at the entry point

Results of thread group

Thread group = 2

Results
Results: Note that as per condition sampler1 is executed for both the thread group but, samplers under if controller are executed only for thread 2.

Module Controller
The Module Controller provides a mechanism for substituting test plan fragments into the current test plan at run-time. A test plan fragment consists of a Controller and all the test elements (samplers etc) contained in it. The fragment can be located in any Thread Group, or on the WorkBench . If the fragment is located in a Thread Group, then its Controller can be disabled to prevent the fragment being run except by the Module Controller. Or you can store the fragments in a dummy Thread Group, and disable the entire Thread Group.

Module Controller contd..

Module to Run: The module controller provides a list of all controllers loaded into the gui. Select the one you want to substitute in at runtime.

As the other controllers are not made deactive so all controllers have been executed once. Under Module controller Simple Controller have been placed. Simple Controller have been executed twice.

Timers

Constant Timers
To pause each thread for the same amount of time Constant timer is used. Timer can be added as a child to thread group or as a child to sampler. If added to thread group it will delay the entire thread group for the time specified. And if it is added to sampler then it will delay the sampler by the given amount of time.

Uniform Random Timer

This timer pauses each thread request for a random amount of time, with each time interval having the same probability of occurring. The total delay is the sum of the random value and the offset value. Total Delay = Offset + Random Delay.

Synchronizing Timer
The purpose of the SyncTimer is to block threads until

X number of threads have been blocked, and then they are all released at once. A SyncTimer can thus create large instant loads at various points of the test plan.

Sync timer results

Synchronization timer(contd):
In the above slide Random timer is kept to ensure that all the threads are delayed by random amount of time. Each thread is delayed by 3 to 13 seconds.
Synchronization timer is used to make sure that all the desired no. of threads hit simultaneously at a link. So that the response of the page can be retrieved. If any thread reaches to the page first it has to wait for the desired no of threads , so that they all can go simultaneously to the page.

Pre- Processors

Pre-Processor
A Pre-Processor executes some action prior to a Sampler Request being made. If a Pre-Processor is attached to a Sampler element, then it will execute just prior to that sampler element running. A Pre-Processor is most often used to modify the settings of a Sample Request just before it runs, or to update variables that aren't extracted from response text.
Some of the Pre Processors 1) HTML Link Parser 2) Counters 3) User Parameters

HTML Link Parser

This modifier parses HTML response from the server and extracts links and forms. A URL test sample that passes through this modifier will be examined to see if it "matches" any of the links or forms extracted from the immediately previous response. It would then replace the values in the URL test sample with appropriate values from the matching link or form

Post Processors

Post Processors
A Post-Processor executes some action after a Sampler Request has been made. If a Post-Processor is attached to a Sampler element, then it will execute just after that sampler element runs. A Post-Processor is most often used to process the response data, often to extract values from it
1) Regular expression extractor
2) XPath Extractor

Regular expression extractor

Results
Regular Expression Extractor

Assertions

Assertions
Assertions allow the ability to assert facts about responses received from the server being tested. Using an assertion, you can essentially "test" your that your application is returning the results you expect it to.
1) Response assertion

2) Size assertion
3) Duration assertion

Response Assertions
Response Assertion

Recording using Jmeter for http request

1. start jmeter with jmeter.bat on windows and jmeter.sh on unix 2. select test plan on the tree 3. right click on the test plan and add a new thread group: add -> thread

group 4. select the thread group 5. right click add -> config element -> Http Request Defaults 6. Protocol enter HTTP 7. Server name enter jakarta.apache.org 8. Path leave blank 9. Port number enter 80 10.select workbench 11.right click on workbench and add the Http proxy: add -> non-test elements -> HttpProxy Server 12.Port field - Enter 8080 Now also set the Browser proxy settings as localhost and port 8080 then start the test from Jmeter

Proxy Server

Browser proxy setting

Listeners

Aggregate Report

Parameters of Aggregate Report

Label - The label of the sample. If "Include group name in label?" is selected, then

the name of the thread group is added as a prefix. This allows identical labels from different thread groups to be collated separately if required. # Samples - The number of samples for the URL Average - The average time of a set of results Median - The median is the time in the middle of a set of results. 50% of the samples took no more than this time; the remainder took at least as long. 90% Line - 90% of the samples took no more than this time. The remaining samples at least as long as this. (90 th percentile) Min - The shortest time for the samples of the given URL Max - The longest time for the samples of the given URL Error % - Percent of requests with errors Throughput - Throughput measured in requests per second/minute/hour Kb/sec - The throughput measured in Kilobytes per second Times are in milliseonds.

Recording from Badboy for Https request

Recording https request from badboy

Badboy test plan after importing to jmeter

Remote Testing

Distributed Test
It is recommended that we should not run more than 200

threads from one machine, so in order to test for 600 or 1000 users we go for distributed testing

Master the system running Jmeter GUI, which controls

the test Slave the system running jmeter-server, which takes commands from the GUI and send requests to the target system(s) Target the web server we plan to load or stress Test

Remote Testing concept

Before we go ahead with distributed testing, there are a

couple of things to check. 1. the firewalls on the systems are turned off. 2. all the clients are on the same subnet. 172.16.50.121, 172.16.50.101.. 3. the server is in the same subnet, if 192.x.x.x or 10.x.x.x ip addresses are used. If the server doesn't use 192 or 10 ip address, there shouldn't be any problems. 4. Make sure jmeter can access the server. 5. Make sure you use the same version of Jmeter on all the systems. Mixing versions may not work correctly.

Remote testing steps

On the slave systems, go to jmeter/bin directory and execute jmeter-server.bat (jmeterserver on unix). On windows, you should see a dos window appear with jre\[version] \bin\rmiregistry.exe. If this doesn't happen, it means either the environment settings are not right, or there are multiple JRE installed on the system. Note: [version] would be the jre version installed on the system.
1. Open jmeter-server.bat in a text editor 2. go to line 44 and find :setCP 3. edit START rmiregistry to the full path. Example: START C:\j2sdk1.4.2\jre\bin\rmiregistry 2. On master system acting as the console, open windows explorer and go to jmeter/bin directory 3. open jmeter.properties in notepad or wordpad 4. edit the line remote_hosts=127.0.0.1 5. add the IP address. For example, if I have jmeter server running on 192.168.0.10, 11, 12, 13, and 14, the entry would like like this: remote_hosts=192.168.0.10,192.168.0.11,192.168.0.12,192.168.0.13,192.168.0.14 6. Start jmeter. 7. Open the test plan you want to use

Distributed Testing architecture

Remote testing

Remote testing
Machine name is different under samplers