You are on page 1of 54

Where Value and Innovation Co-exist

Performance Testing
&
Apache JMeter

Narasimha Nallabothula
ValueLabs
Performance Testing COE

Where Value and Innovation Co-exist

Day 1: Agenda

What is Performance Testing?


Why Performance Testing is required?

Types of Performance Testing


JMeter Overview
JMeter Architecture and working process
Download & Setting up the JMeter Environment
Run the JMeter

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Performance Testing is defined as validating the


Speed
Scalability
Reliability
And stability
of a server, network and application

Discipline which uncovers what components of the


application/environment needs to be improved
before it goes to market

ValueLabs | www.valuelabs.com | Confidential

What is
Performance
Testing?

Where Value and Innovation Co-exist

OK, Now I understand what is


Performance testing.
But why Performance Testing
required to my application ?

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Key Types of Performance Testing


Performance Testing

Spike
Test

Load
Test

Stress Test

ValueLabs | www.valuelabs.com | Confidential

Volume
Test

Endurance
Test

Where Value and Innovation Co-exist

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Load Test
Load Test will be conducted to verify EUTs behavior under normal operational capacity.
o It would follow a ramp-up pattern where the load will be gradually increased. A steady
load state will be maintained for specified time and will be gradually ramped-down.

This test will help us to


o Measure and determine the response times and throughput rates to support peak
load

o Find out resource utilization of the servers under normal production load
o Identify functionality errors under load

Is my application is capable
to handle peak production
load? any errors will occur
under load?
ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Stress Test
Stress Test will be conducted to determine the stability of a given system or component
beyond normal operational capacity, often to find a breaking point of the system

o Initially, a low load will be applied on EUT and it would be constantly ramped up till it
reaches a breaking point (EUT stops responding or a steep hike in response time is
observed or environmental errors are observed because of load)

This test will help us to


o Identify application issues that arise or become apparent only under extreme
conditions

o Determine the maximum sustainable users and throughput of the EUT/AUT


o Identify at what point the system start degrading/fail and reasons for the failure

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Spike Test
Spike Test will be conducted to determine the stability of a given system when there is a
increase/decrease in user load

This test will help us to


o Find out whether the application can handle significant changes in the load
Say for example, test will be initiated
with 100 concurrent user for a certain
period of time, suddenly the concurred
user load will be increased to 1000 and
then to 1500 users. Like wise the user
load will be decreased to 100 concurrent
users and verify the system returns to
normal operation and retaining the
performance as initiated.

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Endurance/Soak Test

Endurance Test will be conducted to verify if the system can sustain continuous
expected load for prolonged period of time

o
o

Endurance Testing is a subset of Load Test


Test will be performed with defined set of concurrent users for a prolonged
period of time

This test will help us to

Discover memory utilization issues and detect potential leaks (memory leaks) in
EUT under sustained use.

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Volume Test
Volume Test will be conducted to test the application performance under varying database
volumes

This test will help us to


o Measure response times, throughput rates and resource-utilization levels of EUT
under different volumes

o Determine how database acts under various circumstances

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Are there any tools


to test the
performance of my
application?

ValueLabs | www.valuelabs.com | Confidential

Yes! we have, I will tell you few tools


Open Source : Apache JMeter, Open STA
Commercial: HP LoadRunner, Silk
Performer, NeoLoad

Where Value and Innovation Co-exist

JMeter Overview

JMeter is an open source testing tool from Apache

Initially Stefano Mazzocchi developed the JMeter, later it is


expanded by Apache

JMeter is a Java Desktop application developed using Java


Swings

JMeter is purely developed by Java, hence it can be run on


any environment that have JVM

It is extendable and flexible to build your own components

JMeter also used to perform functional and automated tests

JMeter store its test plans in XML format. This means you

can customize a test plan using a text editor

It can be configured in any machine/environment easily

ValueLabs | www.valuelabs.com | Confidential

What is this?
Let me
explore

Where Value and Innovation Co-exist

JMeter Architecture And Working Process

Execution Environment

System Under Test

Virtual
Users

JMeter Console, Monitor and Load


Machine

ValueLabs | www.valuelabs.com | Confidential

Statistics and
Performance Data

Where Value and Innovation Co-exist

Download and Install process of JMeter


Pre-requisite

Install JDK 1.6 or above

Installation

Download the binary or source file from the following location


http://jmeter.apache.org/download_jmeter.cgi

Unzip the downloaded file in desired location

Set the windows environment variables


JAVA_HOME: <Its JDK home just like C:\Program Files\Java\jdk1.6.0_18>
PATH: Append <JDK bin folder path like C:\Program Files\Java\jdk1.6.0_18\bin; >
CLASSPATH: JAVA_HOME\lib

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Run / Launch JMeter

By invoking the below files

OS

File Name

Windows

Jmeter.bat

Linux

Jmeter.sh

Mac

Jmeter.sh

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

JMeter Main / Default GUI

The screen contains the following components


o Test Plan - A test plan describes a series of steps that JMeter will execute when run.
o WorkBench - The WorkBench simply provides a place to temporarily store test elements.

ValueLabs | www.valuelabs.com | Confidential

Where Value and Innovation Co-exist

Day 2: Agenda
Elements of the JMeter
Steps to build a script
Steps to replay the script

ValueLabs | www.valuelabs.com | Confidential

18

Where Value and Innovation Co-exist

Threads defines a collection of users


You can control
The no. of users simulated
The ramp-up and ramp-down
patterns
Duration of the test
Start and stop time test

It will execute before a Sampler


executes
Update variable in Sample
Modify the settings of a sampler
Ex: HTML URL Re-writing modifier

It will execute after a Sampler


executes
It is used to Process the response
data (To retrieve a value from the
response)
Ex: RegExp Extractors, Xpath
Extractors, JDBC Post processors

Samplers

Elements of JMeter

To set the configuration values of all


samplers like
Web Server's hostname
Database URL
Cookie Manager, CSV Dataset
Config etc

Threads (Users)

Config Elements

Timers

Pre Processors

JMeter thread sends request without


stopping between each sampler
Timer elements permit us to define a
delay between requests
If timers not defined, JMeter will
attack with nonstop requests
To control the flow of the JMeter
execution
To apply logics/conditions on JMeter
Samplers
To alter the order of the requests

Logical Controllers

Post Processors

Assertions

Listeners
ValueLabs | www.valuelabs.com | Confidential

It simulates a Users request for a


page (URL)
To deliver different types of requests
against to your server
Ex: HTTP Request Sampler, JDBC
Request, SMTP Request,

Assertions will allow you to validate


the response of the requests
To verify that your application is
returning expected results
You can verify some text, response
code, size of the response etc
To view Samplers result in the form of
tables, graphs, trees or simple text
To view and collect the performance
statistics

Where Value and Innovation Co-exist

Flow of a Web Request

Request

Response

End User
System Under Test

Flow of a Web Request with JMeter Proxy


Request

JMeter
Proxy
Response

End User
System Under Test

JMeter Proxy feature allows you to record the traffic / requests simply
o While users navigate/browse the pages through a standard browser, JMeter
will watch and record the traffic

ValueLabs | www.valuelabs.com | Confidential

20

Where Value and Innovation Co-exist

How to Record

(Contd)

Following are the steps to record a HTTP request


1. Right Click on WorkBench Add Non - Test Elements HTTP Proxy Server

ValueLabs | www.valuelabs.com | Confidential

21

Where Value and Innovation Co-exist

How to Record

(Contd)

The HTTP Proxy Server screen is as shown below

ValueLabs | www.valuelabs.com | Confidential

22

Where Value and Innovation Co-exist

How to Record

(Contd)

2. Open the browser, click on Tools Internet Options Connections LAN


Settings
a) Check the Use a proxy server for your LAN

b) Enter Address localhost


c) Port 8080 ( This is the port number that is configured in HTTP Proxy Server,
ensure that both are same)
d) Click on OK button

ValueLabs | www.valuelabs.com | Confidential

23

Where Value and Innovation Co-exist

How to Record

(Contd)

3. Select WorkBench, right click Add Logic Controller


Select Transaction Controller.
4. Rename Transaction Controller to appropriate page title
like Login page, Inbox etc.
5. Select the appropriate Target Controller in HTTP Proxy
Server while browsing the page during recording.
6. Click on Start button in HTTP Proxy Server.
7. Go to the configured browser and access the application
8. Ensure that the Target Controller will contains browsed
requests.
9. Once the recording is done click STOP button in HTTP
Proxy Server.
10. Save the WorkBench by right clicking on it.
ValueLabs | www.valuelabs.com | Confidential

24

Where Value and Innovation Co-exist

How to Replay
1. Right-click on the Test Plan element and select Add Thread Group . Now the Thread
group is created.
2. Thread Group is a pool of users that will execute a particular test case against your server.
Thread Group configurations will be explained during execution .
3. Right-click on the Thread Group element and select Add Config Element HTTP
Cookie Manager. Now the HTTP Cookie Manager is created.
4.

Copy the Transaction from WorkBench and paste to the


Thread Group as Child.

5.

In order to see the status(pass/fail during execution) of each


Transaction, the test plan should have Listener( Listener are
used to view, save, and read saved test results.).

ValueLabs | www.valuelabs.com | Confidential

25

Where Value and Innovation Co-exist

(Contd)

How to Replay
6. Now the test plan contains the Thread Groups, Logic Controllers, Sampler
generating controllers & Listeners.

7. Click on Run, by entering Number of Threads(Users) as 1 in Thread Group.


8. Check the status in View Results Tree, View Results in Table or Summary Report.
a) Status as Passed then it will appear as
b) Status as Failed then it will appear as

ValueLabs | www.valuelabs.com | Confidential

26

Where Value and Innovation Co-exist

Day 3: Agenda
Correlation (Regular Expressions)
Parameterization
And other script customizations

ValueLabs | www.valuelabs.com | Confidential

27

Where Value and Innovation Co-exist

Correlation

When you record a business scenario, JMeter generates a script that contains the actual values
used during recording. When you replay that script it will send the recorded values unless you
change them

Correlation is used to obtain data which is unique for each run

It is useful to extract some information (like session id, timestamps, url links etc) from the
response of a request
During Recording the script in JMeter

Request 1 : Login
Response for Request 1 along with session id 1234567
Request 2: Session ID 1234567
Response for Request 2

User (Browser)
ValueLabs | www.valuelabs.com | Confidential

Server

Where Value and Innovation Co-exist

During Replay the Script in JMeter


Request 1 : Login
Response for Request 1 along with session id 1112223
Request 2: Session ID 1234567
Session failed as 1112223 is expected by the server

JMeter Script

Server
Solution

Request 1 : Login
Response for Request 1 along with session id 999000
Capture the new session id from the above
response and use it in the next request(s)

Request 2: Session ID 999000

JMeter Script
ValueLabs | www.valuelabs.com | Confidential

Successful response to the Request 2

Server

Where Value and Innovation Co-exist

Correlation
1. Right Click the Request Add Post Processors Regular Expression Extractor
2. Regular expressions are used to search and manipulate text.
3. Allows the user to extract values from a server response using regular expression.
4. Regular Expression Extractor extracts the requested values, generate the template
string, and store the result into the given variable name.
5. The Regular Expression Extractor needs to be added in the previous request to
capture the dynamically changing data in the current request.
6. Determine the pattern where the desired text will find.

ValueLabs | www.valuelabs.com | Confidential

30

Where Value and Innovation Co-exist

Correlation

Reference Name: The name of the variable where the dynamically parameterized value get
stored, EX: Data
Regular Expression: This must contain at least one set of parentheses "()" to capture a portion
of the string, left and right boundary of the parameter to be correlated can also be given. The text
groups that will extracted are enclosed by the characters '(' and ')'. For Ex the expression is <td
id="ID">(.+?)</td>\s*<td id="Name">(.+?)</td>\s*, name of the expression is Data , using this
expression ID and Name is captured in the name like Data_1 (ID)& Data_2(Name) variables
ValueLabs | www.valuelabs.com | Confidential

31

Where Value and Innovation Co-exist

Correlation

Template: Each group of text extracted will be placed as a member of the variable Data,
following the order of each group of pattern enclosed by '(' and ')'. Each group is stored as
refname_g#, where refname is the string you entered as the reference name, and # is the group
number. $1$ to refers to group 1, $2$ to refers to group 2, etc. $0$ refers to whatever the entire
expression matches. For Ex the template for the above example is $1$$2$

Match No: Indicates which match to use. The regular expression may match multiple times

Default Value: If no matches are found, then the value given in this field will be substituted

ValueLabs | www.valuelabs.com | Confidential

32

Where Value and Innovation Co-exist

Parameterization

When you record a business process, JMeter generates a script that contains the actual values
used during recording

Lets say, JMeter generated the following recorded post parameters to login into the application

When you replay the script using multiple users and iterations, you do not want to repeatedly
use the same Username and Password values. Instead, you replace the values with a
parameter

The users substitute the parameter with values from a data source that specified

The data source can be either a file, or internally generated variables

ValueLabs | www.valuelabs.com | Confidential

33

Where Value and Innovation Co-exist

JMeter Parameterization
2. Static Value Parameterization
a) User Parameters
b) Function Helper Dialog (_StringFromFile, _CSVRead)
c) Config Elements CSV Data Set Config

ValueLabs | www.valuelabs.com | Confidential

34

Where Value and Innovation Co-exist

Parameterization
User Parameters
1. Right Click the Request Add Pre Processors User Parameters

ValueLabs | www.valuelabs.com | Confidential

35

Where Value and Innovation Co-exist

Parameterization
User Parameters:

1.

User Parameter are used to send the specify values for User Variables specific to individual threads

2.

Values can be accessed in any test component in the same thread group, using the syntax:
${<ParameterName>}.

3.

Make the number of users in Thread Properties to the number of users assigned in the User Parameter.

ValueLabs | www.valuelabs.com | Confidential

36

Where Value and Innovation Co-exist

Parameterization
Function Helper Dialog : _StringFromFile
1. Create a Parameter txt File in desired/specific location .
2. Click Options Function Helper Dialog in JMeter.
3. Choose the function __StringFromFile.

ValueLabs | www.valuelabs.com | Confidential

37

Where Value and Innovation Co-exist

Parameterization
Function Helper Dialog : _StringFromFile
1. Provide full path of the location of file.
2. Provide a name to the variable
3. Provide start file sequence Number as needed.
4. Click on Generate button.
5. Copy and paste the value generated to the value of the variable which has to be

parameterized in the request.


6. The data will be taken from the file while execution. This function can be used to provide
only one column of data.

ValueLabs | www.valuelabs.com | Confidential

38

Where Value and Innovation Co-exist

Parameterization
Function Helper Dialog : _CSVRead
1. Create a CSV File with the parameters in desired/specific location.
2. Click Options Function Helper Dialog in JMeter.

3. Choose the function __CSVRead.

ValueLabs | www.valuelabs.com | Confidential

39

Where Value and Innovation Co-exist

Parameterization
Function Helper Dialog : _CSVRead
1. Provide full path of the location of file.
2. Provide the column number as needed.
3. Click on Generate button.
4. Copy and paste the value generated to the value of the variable which has to be
parameterized in the request.
5. Column number will start from 0
6. The data will be taken from the file while execution.
Note: StringFromFile function can be used to provide only one column of data, using
_CSVRead & CSV Data Set Config has facility to send multiple column data during
execution.

ValueLabs | www.valuelabs.com | Confidential

40

Where Value and Innovation Co-exist

Parameterization
Config Elements CSV Data Set Config
1. Create a CSV File with the parameters in any location .
2. Right Click Test Group Add Config Element CSV Data Set Config.

ValueLabs | www.valuelabs.com | Confidential

41

Where Value and Innovation Co-exist

Parameterization
Config Elements CSV Data Set Config
1. Name : Descriptive name for this element that is shown in the tree. .
2. Filename : Name of the file to be read
3. File Encoding : Generally its blank for windows
4. Variable Names: List of variable names (comma-delimited)
5. Delimiter: Delimiter to be used to split the records in the file, generally keep, for CSV file
6. Recycle on EOF: Should the file be re-read from the beginning on reaching EOF? (default is true)
7. Stop thread on EOF?: Should the thread be stopped on EOF, if Recycle is false? (default is false)
8. Sharing mode: How to share the CSV file to current thread/ Current thread group / Current thread
9. Values can be accessed in any test component in the same thread group, using the syntax:
${<ParameterName>}

ValueLabs | www.valuelabs.com | Confidential

42

Where Value and Innovation Co-exist

Day 4 : Agenda

Steps to build a Performance Test Plan in JMeter


Run a performance Test in JMeter
Analyze and prepare the report
Demo on JMeter Distributed Testing
Sample Report walkthrough

ValueLabs | www.valuelabs.com | Confidential

43

Where Value and Innovation Co-exist

Test Execution
1.

Thread Group defines a pool of users that will execute a particular test case against your server .

2.

Following are the items in the Thread Group


a.

Name: Name of the thread group

b.

Action to be taken after a Sampler error: If a sampler error occurs select any one option below as per
requirement
i.

Continue - ignore the error and continue with the test

ii.

Start Next Loop - ignore the error, start next loop and continue with the test

iii.

Stop Thread - current thread exits

iv.

Stop Test - the entire test is stopped at the end of any current samples.

v.

Stop Test Now - the entire test is stopped abruptly. Any current samplers are interrupted if possible

c.

Number of Threads: Number of users to simulate

d.

Ramp-up Period: How the number of users to start the action.

e.

Loop Count: Number of times to perform the test case

f.

Start Time & End Time: This is used to when to start the test and when to end.

g.

Duration (seconds): Total duration of the execution.

h.

Startup delay (seconds): Time delay to start the test.

ValueLabs | www.valuelabs.com | Confidential

44

Where Value and Innovation Co-exist

Test Execution Single Scenario

Select Thread group Add Transaction Controller.

Rename the Transaction Controller to appropriate name if transactions are defined in WorkBench.

ValueLabs | www.valuelabs.com | Confidential

45

Where Value and Innovation Co-exist

Test Execution Single Scenario

Move/copy the Samplers to the respective Transaction Controller.

Add Listeners to the Thread Group.

Provide the following data in Thread group

Number of Threads

Action to be taken after a Sampler error.

Ramp-up Period.

Loop Count.

Click on Start under Run or Start Run image.

ValueLabs | www.valuelabs.com | Confidential

46

Where Value and Innovation Co-exist

Test Execution Multiple Scenario

Right click on Test plan Select Merge Select the another script.

ValueLabs | www.valuelabs.com | Confidential

47

Where Value and Innovation Co-exist

Test Execution Multiple Scenario

Add Listeners to the Test Plan.

Provide the following data for each in Thread group

Number of Threads

Action to be taken after a Sampler error.

Ramp-up Period.

Loop Count.

Duration

Start up delay

Click on Start under Run or Start Run image.

ValueLabs | www.valuelabs.com | Confidential

48

Where Value and Innovation Co-exist

Test Results

Thread Group 1 load criteria is as shown in the screen below

ValueLabs | www.valuelabs.com | Confidential

49

Where Value and Innovation Co-exist

Test Results

Thread Group 2 load criteria is as shown in the screen below

ValueLabs | www.valuelabs.com | Confidential

50

Where Value and Innovation Co-exist

Test Results View Results Tree

View Results tree is used for debugging for the parameters that are substituted and the correlated value is passed in
the post request.

Add the View Results Tree listener to the Test plan/Thread group.

ValueLabs | www.valuelabs.com | Confidential

51

Where Value and Innovation Co-exist

Test Results View Results in Table

View Results Table contains Start time, Label, Response time(ms) and Bytes transferred

Add the View Results Table listener to the Test plan/Thread group

ValueLabs | www.valuelabs.com | Confidential

52

Where Value and Innovation Co-exist

Test Results View Summary Table

Summary Report shows Average, Minimum and Maximum response times along with Std deviation , Error % and
throughput

Add the Summary Report listener to the Test plan/Thread group

ValueLabs | www.valuelabs.com | Confidential

53

Where Value and Innovation Co-exist

Thank you !

narasimha.nallabothula@valuelabs.com

Extension : 32167

You might also like