You are on page 1of 10

20% 20%

50 20% 20%

20%
40
How to deal with the"Necessary Evil" 20%
20% 20%

30
of

TESTING METRICS
20%
20
20%

20%
10
By 20%

PLANNING AND MAINTAINING


A HELPFUL METRICS PROGRAM

20%

WWW.PRACTITEST.COM
Why should you care about
Testing Metrics?

Metrics are a powerful way to express data, project progression,


ROI and anything else you want to present. Often, metrics is what
we and other project stakeholders rely on when making decisions,
so they need to be handled with care.

We are all required to report our work efforts on a regular basis, as


a means to communicate project progression status. Many times
the test management tools we use produce reports automatically
and we just use whatever default reports it spits out and then
distribute them to all involved, but that is a big mistake!

Metrics are in fact a powerful tool that can be used for good or for
bad, much like a weapon. The metrics and reports you produce are
trusted and used to make actionable decisions, which, can make
or break any project. That is why it's important to provide metrics
responsibly.

This ultimate guide to Testing Metrics will take you through the
pros and cons of metrics, and how to plan and maintain a useful
metrics program that can serve real value.

WWW.PRACTITEST.COM
- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

Table of contents:

The Pros of Metrics ....................................................1


The Cons of Metrics ...................................................2
The Laws of (Conservation of) Metrics .....................3
Planning a Metrics program.......................................4
Maintaining a Metrics program..................................6
Tips and tricks for working with Metrics...................7

WWW.PRACTITEST.COM
share this:
- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

Does your team have a standard set of metrics in use?


Have these metrics caused more harm than help?
Metrics like knives are a great tool that can also be
dangerous in the wrong hands.

The PROS
Effective means of communication -
Not only for communicating status, but also for conveying
what is important and where effort is needed

Usually clear -Good metrics are easy to understand by


people both in and outside the team

Benchmark and compare more objectively


Between teams, processes, versions and more

Track progress - See how things change over time, make


50
educated predictions and corrections as you go.

Easy to distribute -Share or 40


send simple spreadsheets, chart
& reports by emails, or designated online dashboard displays.
30
People like them - They are compact and should be eye-
pleasing ways of communicating
20 information.

10

-1-
0
WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

Metrics: TheCONS

People don't like being measured - Do you like having


your productivity measures by the amount of hours
you stay at the office? We didn't think so....

Present only "one-side" of the story -


Metrics can be contorted into showing anything you
want, so you end up seeing one version of the truth and
not the whole picture.

Metrics are going to be used by managers as well, who


might -
Jump to conclusions without waiting to listen to all
the information and will try to apply quick fixes to
50 those issues behind
complex issues, while "hiding"
graphs and tables
40

30

20

10

-2-
0
WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

Accept the:
"LAWS OF (CONSERVATION OF) METRICS"

1. Anything that is measured will improve over time

2. Whenever something improves, it will be at the expense of


something else that will worsen accordingly

3. You can only measure 5 (sometime only 3) aspects of anything.


Trying to measure more will leadto confusion and failure

4. Metrics will become more ineffective after a number of iterations


(kind of like the Pesticide Paradox)

50

40

30

20
-3-

10
WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

PLANNING A METRICS PROGRAM


1. Define the objectives of your metrics

Who is your audience?


Management? your team?
your users? regulation or HR?

Each audience has different interests regarding project QA.

Your managers might care more about time and


budget spend on testing and what value that has
brought, while HR might care more about team
productivity, and your users would care more about
when the latest feature update for instance will be
released.

Customize your metrics to your audience. Each of them


need to see different information to make different
decisions

-4 -

WWW.PRACTITEST.COM
WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

PLANNING A METRICS PROGRAM

2. Make sure you have the right metrics


Ask yourself the 5 Why's -
To peel away the layers and reach your core metrics

3. Make sure you build your metrics correctly


Be S.M.A.R.T - Metric need to be:

Simple - Clear and easy to follow


Measurable - Numbers and Statistics over descriptions
Actionable - Propose Solutions & not only Point at Flaws
Repetitive - Show information over versions and products
Timely - Provide the information when it is still relevant

Some of your most important metrics may be available ONLY


after a project is done (external to the testing process) but should
be included:
Escaping defects, Rejected Defects, Server Exception Levels,
Support calls post release etc.

-5 -

WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

MAINTAINING A METRICS PROGRAM

1. Metrics need to be constantly reviewed for:


- Accuracy
- Relevancy

Learn to adapt - As your project, technology and


people change so will your metrics.

50
2. Don't get too comfortable!
40 need to change your metrics once in a while, bugs and
You
people learn to work around them.
30

3.
20 Stay Agile - even with your metrics:

Be ready to change metrics, even half way during your project


10

0
Item 1 Item 2 Item 3 Item 4 Item 5
Testing is a service, and you are a service provider.
Therefore, your metrics plan should be of service as well.
Understand who is the client of your service,
and remember to serve his/her needs, not yours!

-6 -

WWW.PRACTITEST.COM
share this:

- PLANNING AND MAINTAINING A HELPFUL METRICS PROGRAM -

TIPS AND TRICKS FOR


WORKING WITH METRICS

Presentation is as important as content

Have different metrics for different people

Capture more data than you need today, you may need it later.

Provide analysis of your results with every graph


(dont trust the people to understand what they see)

Make sure to provide multiple angles on the same data.

Too many metrics can confuse - tell a story, but keep it interesting.

Push metrics, dont make people ask for them since it wont be
relevant by the time you provide them.

For more helpful resources visit the QA learning center:


www.practitest.com/qa-learningcenter
-7 -

WWW.PRACTITEST.COM

You might also like