Load Testing with JMeter

Larry Davidson
October 21, 2008
What is Load Testing?
2
User User User User

App

Normal Use – Few Concurrent

User User User User User User User User User User User User User User User User User

App

Spike Use – Many Concurrent

User User User User User User User User User User User User User User User User User

App

Spike Use – Many Concurrent

Load Testing
Key goals
Can app support expected concurrent users?
At what load does the app break?
Steps
Identify primary user paths
Identify expected concurrent users
Both now and down the road
Set up virtual users to hit the app
Run the test
Analyze the results
Load Testing Approaches
Open Source Tools
JMeter
ab
http_load
The Grinder
Siege
Commercial Tools 7
LoadRunner (around $200,000 to own!)
WebLoad
Third-party $ervices
Why We Chose JMeter for Internal Testing

Easy to install and use

Free!
Java — most platforms
GUI vs. command line
Just download and run!
Feature-rich
Post forms 8
Record from browser
Load test data from files
Add logic, variables & functions
Run one test from multiple machines
Test many protocols, not just HTTP
Installing & Running JMeter
1. Download from http://jakarta.apache.org/jmeter 
2. Unzip (path with no spaces preferred)
3. Go to "bin" directory
4. Run appropriate file
Windows:  Double-click jmeter.bat
Mac:          Double-click ApacheJMeter.jar
9
Unix/Mac:  Run ./jmeter from command line
Looks complicated ... but it's not!
10
Using JMeter's GUI
Hierarchy based 
"Test Plan" is root element
Add test elements as children
Right-click and choose "Add > ..."
Give items meaningful names
"WorkBench" is for non-test helpers
HTTP Proxy recording 11

Modify properties in right panel

Properties saved after leaving entry fields
What We're Trying to Do
Simulate users
Defined numbers of concurrent users
Defined period of time
Various user types (normal, registered, admin, etc.)
Simulate web requests
Specific web pages
Form submissions 12
Alternate or random paths
Analyze reports
Response times acceptable?
Handles expected load?
What load causes failure?
JMeter's Nomenclature
User                   Thread

Request             Sampler

Report                Listener

13
Let's run a Load Test already!
14
Setting Up Your First Load Test
Simulate users
Add > Thread Groups
Number of Threads (concurrent users) = 10
Rampup Time (seconds until users active) = 1
Loop Count (how many repetitions) = 1

Simulate web requests

Add > Sampler > HTTP Request
15
Server Name or IP = www.yahoo.com
Path = /

Analyze reports
Add > Listener > Aggregate Graph
Add > Listener > View Results Tree 
Running Your First Load Test
Choose "Run > Start" (ctrl-R)
Click on the "Aggregate Graph"
Data fills in as test runs
Click "Display Graph" for chart 47 threads (“users”) Running!

Watch upper right square

Green = running, Gray = done
47/100 means 47 of 100 threads are running 16

To run again with clean data...

Choose "Run > Clear All" (ctrl-E)
Choose "Run > Start" (ctrl-R)
Verify Details with View Results Tree
Details for each request, good for troubleshooting
Sampler Result
Thread number and details
Response times
Server response codes and headers
Request
Request sent to web server 17

Response Data
Server response body (HTML, etc.)
View as text or rendered HTML, etc. 
Let's get more advanced!
18
A Test with Three User Types
Users can do various things on a site
Good to simulate several common tasks at same time
For this demo, we’ll test 3 users
1. Yahoo Viewer
2. Google Viewer
3. CNN Viewer
To start, add 3 thread groups under Test Plan 19

Set to run all concurrently (this is default)

Add HTTP Request to each, with appropriate URLs
Add Listener under Test Plan and Run
Notice that all 3 URLs are requested at once
Logic Controllers
Group Items
Simple
Choose One Per Loop or Thread Group
Interleave, Random, Switch*
 Looping
Loop, ForEach*, While*
20
Logical*
If
Time Control
Throughput, Runtime 
Randomly Selecting From 3 URLs
Let’s modify the 3 user types Test Plan
Move the 3 requests under one Thread Group
1. Yahoo
2. Google
3. CNN
Choose all 3, right click, choose “Insert Parent...”
Choose “Controller > Random Controller” 21

Add Listener under Test Plan and Run

You’ll see that only one random request is chosen
Add an If Controller
Let’s modify the existing Test Plan
Add “Controller > If Controller” under Thread Group
Copy the 3 requests
1. Yahoo
2. Google
3. CNN
22
Paste under the “If Controller”
Delete the “Random Controller”
Let's make a recording!
23
Recording your load test
Add Proxy Server to WorkBench
“Non-test Elements > HTTP Proxy Server”
Choose a "Target Controller" for recorded data 
Click "Start"
Set browser to use Proxy Server
Edit browser's Network Settings
Usually under Preferences > Advanced > Network/Proxy 24
Proxy = localhost; Port = 8080

Browse and record

Click "Stop", then edit recorded requests
Recording tips and tricks
Limit what's recorded
Enter "text/html" in Content-Type filter (Include)
Enter *.gif, *.css, etc. in URL Patterns to In/Exclude 
Use Grouping if recording multiple pages
Add HTTP Request Defaults under Controller
Fill in domain
Now matching recorded domains will be blank 25

Record time between page requests

Add "Constant Timer" under Proxy 
Enter "${T}" in delay field to record actual time
Recording forms
JMeter records all submitted fields
Easier than examining source code or Firebug!
JMeter also records browser headers
Some pages (e.g., Google) require valid browsers
Remember to accommodate cookies if needed!
add “Config Element > HTTP Cookie Manager”
26
Edit field values after they're recorded
OR ... why not simulate varying user data!
Let's add variables!
27
JMeter Variables
Define global variables
Test Plan (root element)
Config Element > User Defined Variables
Define virtual user variables
Preprocessors > User Parameters
Config Element > CSV Data Set Config
Reference as ${variableName} 28

Can be used in nearly any property

Scope determined by location in hierarchy
Variables from a CSV file
Create a CSV file with the data you want to use
No label row; save in same folder as .jmx file
Add "Config Element > CSV Data Set Config"
Add under the Thread Group that will use it
Set Filename = filename.csv
Set Variable names = username,password (e.g.)
Set variables in HTTP Request "Send Params" 29
Name = username, Value = ${username}
Name = password, Value = ${password}

Will read in data with each new thread, or with loops

Debugging
Add Debug Sampler
Set JMeter variables as "True", others "False"
Add View Results Tree Listener
Run
Click View Results Tree
Click Debug Sampler in Results Tree 30
Click "Response data" tab to view variables
Looped user vars will show with numbers
JMeter Functions
Referenced as ${__functionName(properties)}
Like variables, can use in JMeter properties
Use Options > Function Helper Dialog
Choose a function
Enter parameter values
Click "Generate", then copy/paste function
Helpful functions 31

${__javaScript(expression to evaluate)}
${__threadNum()}
${__Random(min, max, varname)}
${__regexFunction(regEx, template, match, ...)}
So what about reports?
32
Reading Reports
Most times are reported in milliseconds
Throughput
Requests per second
90% Line
Value below which 90% of request times fall
Standard Deviation
33
A measure of certainty / uncertainty
Measures variation from average
Latency
Delay between request and response
Response Time Goals to Aim For
0.1 second — perceived instant
1.0 second — uninterrupted flow
10 seconds — user loses interest
Going further...
35
Higher loads with multiple "slave" computers

Install JMeter on other computers

Find the IP addresses of the other computers
Start JMeter server on all computers
Windows: Double-click [jmeter]/bin/jmeter-server.bat
Mac command line: cd [jmeter]; ./jmeter-server
Add IP addresses to master computer's JMeter
36
Open [jmeter]/bin/jmeter.properties in a text editor
Add IPs to remote_hosts=127.0.0.1,192.168.1.241
Choose "Run > Remote Start All"
Limitations of JMeter
Can't run embedded Javascript, Flash, etc.
But ... use recording to capture those requests
Limited by your computer's network connection
You may have to work with IT to optimize
1 computer limited to about 300 concurrent users
But... use multiple slave computers to handle more
37
 Reports are a bit lacking
But... save results and import into Excel, etc.
Reference

http://jakarta.apache.org/jmeter

38