Automated

performance testing
using Maven & JMeter
George Barnett, Atlassian Software Systems
@georgebarnett
• Create controllable JMeter tests
• Configure Maven to create a repeatable cycle
• Run this “build” in your CI server every 8 hours
• Find performance regressions faster
 WHY?
• Catch regressions quickly
• Stop hunting for bugs
• Code with confidence
• Know the performance cost of features
• Fearless external dependency upgrades
• Go home on time
 ATLASSIAN

• 15,000+ customers
• 100+ developers
• 8 software streams
• 1 performance engineer
 ATLASSIAN

• JIRA - Issue management

• Confluence - Enterprise Wiki
• Bamboo - Continuous Integration
• FishEye, Crucible & Clover - Dev Tools
• Crowd - Single Sign-On.
 BEFORE WE BEGIN
• Continuous Integration is critical
• Keep code working with Unit and Functional
tests
• Broken or irregular builds means no code to
performance test
• Performance testing is NOT a replacement
for other testing (but it helps)
 PROCESS
• Write code (1-6 months)
• Compile code
• Send it to QA (or Production?!)
• Performance Regression
• PANIC!
No performance testing!
 A BETTER WAY
• Write code
• CI compiles code and runs tests
• Send it to QA (or Production?!)
• Performance Regression
• PANIC!
No performance testing!
 A BETTER WAY 2.0

• Performance test artifacts as soon as they

pass Unit and Functional testing
• Fast feedback for developers
• No Surprises
 TOOLS

• JMeter
• Maven & plugins
• Bamboo & plugins
• A Profiler
 JMETER

• Apache Project
• Java
• Supports HTTP, JDBC, SOAP, XML-RPC,
FTP, SMTP, LDAP, JUnit & more.
 MAVEN
• Apache Project
• Java
• Build Tool
• Dependency management
• Documentation
 MAVEN CONCEPTS
• Project Object Model (POM)
• Standardised directory structure
• Build Lifecycle
• Reporting Lifecycle
• Plugins
 MAVEN POM
<project>
<groupid>com.atlassian.com.performance.jira</groupid>
<artifactid>automated-performance</artifactid>

<properties>
<test.dataset>/opt/dataset/dataset1.zip</test.dataset>
</properties>

<build>
<!--Main Lifecycle plugins configured here-->
</build>

<reporting>
<!--Reporting Lifecycle here-->
</reporting>
</project>
 MAVEN PLUGINS
• Chronos
• Ant-run
• Cargo
• Maven Assembly plugin
• Maven Dependency plugin
 BAMBOO

• Continuous Integration
• Can run builds on Amazon’s EC2
• JMeter Aggregator Plugin
 A PROFILER
• Java Profiler
• CPU Snapshots
• GC & Memory information
• Thread usage and contention
• Triggers & offline sampling
 STEP 1
Package the JMeter test

• Maven Assembly plugin

• Create a .zip from project directories
• `mvn deploy` to repository as a SNAPSHOT
 STEP 2
Configure the Maven automated build

• Package application state into zip files

• Ant-run - application pre-setup
• Cargo - deploy container
• Chronos - configure the JMeter tests
 MAVEN PROFILES
<project>
..
<properties>
<test.dataset>/opt/dataset/dataset1.zip</test.dataset>
</properties>

<profile>
<id>dataset1</id>
<properties>
<test.dataset>/opt/dataset/dataset1.zip</test.dataset>
..

<profile>
<id>dataset2</id>
<properties>
<test.dataset>/opt/dataset/dataset2.zip</test.dataset>
..
 MAVEN PROFILES
• Invoked on command line
• eg: mvn verify -Pmysql, tomcat5, dataset1

• Configuration in profile is added to build

• Use for a wider variation in test
environments
• Use profiles for individual “work units”, eg
JMeter
 ADDING A PROFILER
$ export LD_LIBRARY_PATH=”$JPHOME/bin/linux-x64/

<global.jvm.args>-Xms2000m -Xloggc:${project.build.directory}/gc.log</global.jvm.args>
<jvm.args>${global.jvm.args}</jvm.args>

<profile>
<id>jprofiler</id>
<properties>
<jvm.args>-agentlib:jprofilerti=offline,id=${jprofiler.profileid},config=${jprofiler.configfile} -
Xbootclasspath/a:${jprofiler.home}/bin/agent.jar ${global.jvm.args}</jvm.args>
</properties>
</profile>
 REPORTING
CHRONOS

• Available in maven
• eg: mvn site / mvn chronos:report

• Generates graphs and a numerical report

• Can compare to historical data
• Static :(
REPORTING
CHRONOS
REPORTING
CHRONOS
REPORTING
CHRONOS
REPORTING
CHRONOS
REPORTING
CHRONOS - HISTORY
REPORTING
CHRONOS - HISTORY
 REPORTING
BAMBOO JMETER AGGREGATOR

• Open source Bamboo plugin (supported!)

• James Roper (Atlassian) 20% time project
• Can graph JMeter .JTL and CSV files
• Dynamic
 REPORTING
BAMBOO JMETER AGGREGATOR
 TIPS
• Allocate time for fixing tests
• Your tests WILL break
• Work out a process for fixing bugs
• Gather as many artifacts from each test run
as possible
• React sooner rather than later
 Q&A
gbarnett @ atlassian . com

@georgebarnett

Code samples online soon!

blogs.atlassian.com/developer

Meet Atlassian @ The Summit:

May 31 - June 2 in SF
www.atlassian.com/summit