REPUBLIC OF SLOVENIA

MINISTY OF THE ENVIRONMENT AND SPATIAL

PLANNING
SLOVENIAN ENVIRONMENT AGENCY

WMO Training Workshop on Metrology for

RA III & IV Spanish speaking countries

Buenos Aires, Argentina, 2nd– 6 th November 2015

Interaboratory comparison
and field verification

M.Sc. Drago Groselj

Slovenian Environment Agency

Vojkova 1b, 1000 Ljubljana, Slovenia
TEL: +386 1 4784 100
FAX: +386 1 4784 052
E-MAIL: drago.groselj@gov.si
 Contents

Proficiency testing / Interlaboratory comparison

1) Terms and definition

2) Purpose of interlaboratory comparison
3) Interlaboratory comparison protocol
4) Evaluation of the ILC results
5) Interlaboratory comparison example

Field verification

1) Traceability
2) Field verification kit specifications
3) Example on field verification kit use

Training Workshop on Metrology for RAIII & RAIV

2
 Various titles

 Interlaboratory proficiency study or test

 Interlaboratory comparisons
 Round test, round robin test
 Test of qualification
 Certification study
 Interlaboratory validation

Training Workshop on Metrology for RAIII & RAIV 3

 Proficiency testing - PT

 Evaluation of participant performance against pre-

established criteria by means of interlaboratory
comparisons:
– Quantitative schemes (quantify one or more measurands of the
proficiency test item)
– Qualitative schemes (identify or describe one or more characteristics of
the proficiency test item)
– Sequential scheme (one or more proficiency testing items are distributed
sequentially for testing or measurement and returned to the proficiency
testing provider at intervals for rechecking)
– Simultaneous scheme (involve randomly selected sub-samples from a
source of material being distributed simultaneously to participants for
testing)

Training Workshop on Metrology for RAIII & RAIV 4

 Proficiency testing - PT

– Single occasion exercise (Where proficiency test items are

provided on a single occasion)
– Continuous scheme (items are provided at regular intervals
on a continuing basis (example 3 times per year))
– Sampling (example where individuals or organizations are
required to take samples for subsequent analysis)
– Data transformation and interpretation (Where sets of
data or other information are furnished and the information is
processed to provide an interpretation (or other outcome)

Training Workshop on Metrology for RAIII & RAIV 5

 Interlaboratory comparison – ILC

 Organization, performance and evaluation of

measurements or tests on the same or similar items
by two or more laboratories in accordance with
predetermined conditions
– NOTE – In some circumstances, one of the laboratories
involved in the intercomparison may be the laboratory
which provided the assigned value for the test item

Training Workshop on Metrology for RAIII & RAIV 6

 Interlaboratory comparison purpose

– evaluation of the performance of laboratories for specific

tests or measurements and monitoring laboratories’
continuing performance;
– identification of problems in laboratories and initiation of
actions for improvement which, for example, may be
related to inadequate test or measurement procedures,
effectiveness of staff training and supervision, or
calibration of equipment;
– establishment of the effectiveness and comparability of
test or measurement methods;
– provision of additional confidence to laboratory
customers;
Training Workshop on Metrology for RAIII & RAIV 7
 Interlaboratory comparison

– identification of interlaboratory differences;

– education of participating laboratories based on
the outcomes of such comparisons;
– validation of uncertainty claims;
– evaluation of the performance characteristics of a
method – often described as collaborative trials;
– assignment of values to reference materials and
assessment of their suitability for use in specific
test or measurement procedures;

Training Workshop on Metrology for RAIII & RAIV 8

 Interlaboratory comparison

– support for statements of the equivalence of

measurements of National Metrological Institutes
through "key comparisons" and supplementary
comparisons conducted on behalf of the International
Bureau of Weights and Measurement (BIPM) and
associated regional metrology organizations.
– The International Standard ISO/IEC 17011 requires
accreditation bodies to take account of laboratories’
participation and performance in proficiency testing

Training Workshop on Metrology for RAIII & RAIV 9

 What is an ILC?

 Interlaboratory comparison serves as a tool for comparison

of measurement results (calibration)carried out by
accredited or non-accredited calibration laboratories in the
relevant field of measurement.
 ILC represents very effective
means to demonstrate
technical competence of the
Lab7 Lab8 Lab9

participant a technical base

Lab6 Lab
10
Lab5

for accreditation
RIC Lab
Meteo France 11

It does not provide Lab4 Lab

12
RIC ARSO
traceability for the Lab3
Ljubljana RIC Bratislava

participants laboratory. Lab

Lab
13
Lab2 Lab1 14
Training Workshop on Metrology for RAIII & RAIV 10
 How a typical ILC is organized?

 Majority of ILCs in Meteo community

were organized for basic meteorological
parameters (T, RH, p).
 Set of transfer standards with
appropriate metrological properties (ILC
items) must be chosen.
 It is forbidden to make any adjustments
of the items.

Training Workshop on Metrology for RAIII & RAIV 11

 Reference laboratory
 Reference Laboratory organizes ILC:
• ILC coordination and documentation
• Instructions on handling items, including safety
• Packaging and transportation
• Data analysis and records, evaluation of the results
• Environmental conditions
• Communication with participants
• Confidentiality of records, if agreed among participants
 Participant should be advised to perform measurements in
accordance with their routine procedures.
 Reference Laboratory provides ILC protocol carrying information
on purpose of the ILC, data acquisition, calibration points,
transportation requirements, schedule, reporting of the results.
Eexample: ILC Protocol.doc

Training Workshop on Metrology for RAIII & RAIV 12

 Reference value

ILC often consists of comparison of measuring results of a laboratory

and a reference value defined by reference laboratory, which is the
highest authority for the particular measurements:
• reference value could be defined by reference laboratory
• reference value could be defined by a group of participating
laboratories. The assigned value is calculated as mean of reference
laboratories. The uncertainty of the assigned value is calculated as
uncertainty of mean, with uncertainties of reference laboratories at
each calibration point.

Training Workshop on Metrology for RAIII & RAIV 13

 Evaluation of the results

Number of methods for evaluation of ILC results:

xlab  xref
Z-score: Z Z  2;2 
p
xlab  xref
zeta-score:   zeta  2;2 
u 2
lab u 2
ref

xlab  xref
En-score: En  En  1;1 
U 2
lab U 2
ref

Xlab = value reported by laboratory

Xref = reference value
p = target value of standard deviation

Training Workshop on Metrology for RAIII & RAIV 14

 Normalised error - En
The evaluation of the measurement results was made on
the basis of the normalized error En. The normalized
error En is given by the formula:
xlab  xref
En 
2
U lab  U ref
2
where:
• Xlab - value measured by the participating laboratory,
• Xref - reference value,
• Ulab - expanded (k=2) uncertainty of the value measured by the
participating laboratory,
• Uref - expanded (k=2) uncertainty of the reference value.
If En≤1, the measurement result is assessed as
satisfactory, if EN>1, the measurement result is assessed
as unsatisfactory.
Training Workshop on Metrology for RAIII & RAIV 15
 ILC between RICs

 Organised by RIC Melbourne in 2008 for T, RH and p using

transfer standard (Slovakian RIC, France RIC, Australian RIC
and Slovenian RIC)
• Test items: HM70 (RH), Instrulab 4312 Pt100 (T), PTB220A (p)
• Calibration points:
T [°C]: -40,-30, -20, -10, 0, 10, 20, 30, 40, 50
RH [%]: 10, 20, 30, 40, 50, 60, 70, 80, 90, 95
p [kPa]: 80, 85, 90, 95, 100, 105
• Criteria for evaluation was normalised error En RIC
Melbourne

Training Workshop on Metrology for RAIII & RAIV 16

0.30

France Australie Slovakia Slovenia

0.25

0.20

0.15

0.10

0.05

0.00
4
-0.05

3
-0.10

2
-0.15

0,008
Correction SLOVAKIA (°C) 0
0,007 0 20 40 60 80 100 120
Correction SLOVENIA(°C)
0,006 -1
Correction MF (°C)
0,005
-2

0,004
-3
0,003
France salt France 2 temp Slovakia Slovenia
0,002 -4

0,001

0 CONCLUSION:
-60 -40 -20 0 20 40 60
-0,001 the RICs of RA VI are operating within their stated
uncertainties with respect to temperature, pressure
-0,002
and humidity;

Training Workshop on Metrology for RAIII & RAIV 17

 ILC in South Eastern Europe
 Organised by RIC Ljubljana in 2008 for T, RH and p using transfer
standard (Reference laboratory was Slovenian NMI).
 13 NMHSs were invited; 4 labs participated.
 Test items: TwD Patton (T), HMP45D (T, RH), DTS12G (T), PTB220
(p).
 Confidentiality was agreed.
 Calibration points:
T [°C]: -30, -20, -10, 0, 10, 20, 30, 40
RH[%]: 10, 20, 30, 40, 50, 60, 70, 80, 90, 95
p[kPa]: 80, 85, 90, 95, 100, 105, 110
 Criteria for evaluation was normalised error En.

Training Workshop on Metrology for RAIII & RAIV 18

 Temperature

Manufacturer: Vaisala
Type: DTS12G
Serial number: V38505
0.15
Lab100
Correction [°C]

Lab101
0.10 Lab102
Lab103
0.05 ref

0.00

-0.05

-0.10

-0.15

-0.20
-45 -40 -35 -30 -25-20 -10
-15 0
-5 105 2015 30 25 40 35 50 45 55
Temperature [°C]

Training Workshop on Metrology for RAIII & RAIV 19

 Relative humidity
Manufacturer: Vaisala
Type: HMP 45D
Serial number: V3850021
6.00
Lab100
Correction [%]

5.00 Lab101
Lab102
4.00
Lab103
3.00
ref
2.00

1.00

0.00

-1.00

-2.00

-3.00

-4.00

-5.00
5 10 1520 2530 40
35 50
45 6055 70 65 80 75 90 85 95 95 105
Relative humidity [%]

Training Workshop on Metrology for RAIII & RAIV 20

 Pressure
Manufacturer: Vaisala
Type: PTB220 ACA2A3A1AB
Serial number: A4610018 - CELL 1
0.50
Correction [hPa]

Lab101
Lab102
0.40
Lab103

0.30 ref

0.20

0.10

0.00

-0.10

-0.20
750 800 800 850 900
850 950
900 1000
950 1050
1000 1100
1050 1100
Pressure [hPa]

Training Workshop on Metrology for RAIII & RAIV 21

 Field verification kit

• Quality assurance for on-site instruments suitable

also for NMHSs without calibration labs
• Field verification is not calibration, but test,
verification
• Verification are made on site under environmental
conditions – limited range
• Transfer standards for temperature, humidity and
pressure, precipitation, solar radiation, wind speed
• Suitable for traditional and AWS instruments
• Kit must be periodically recalibrated in laboratory

Training Workshop on Metrology for RAIII & RAIV 22

 Ground stations traceability

NMI, RIC, NMI, RIC,

accreddited laboratory Accredited Laboratory

External traceability

Internal traceability
NMHS Calibration
laboratory
Field verification kit

Field verification kit

Ground stations Ground stations

Training Workshop on Metrology for RAIII & RAIV

 Specification of field verification kit - example

• Robust instruments with display

• Battery powered with autonomy up to 8h

Barometer with digital display To measure barometric pressure Thermometer with digital display To measure temperature
Output Digital display and RS232 Output Digital display
Operating temperature range -10 to 40°C Power supply Rechargeable battery pack
Power supply Rechargeable battery pack Operation time using battery >8h
Operation time using battery >8h
Measurement range -40 ... +60 °C
Resolution (display) 0.01hPa Temperature sensor Pt100 RTD IEC751 1/3 Class B
Measurement range 600 ... 1100 hPa Resolution 0.1°C
Linearity <0.1 hPa
Hysteresis <0.1 hPa
Temperature dependence < 0.1 hPa
Total accuracy <0.2 hPa
Long term stability < 0.15 hPa/year

Hygrometer with digital display To measure relative humidity

Output Digital display
Power supply Rechargeable battery pack
Operation time using battery >8h

Measurement range 0 ... 100 % RH

Resolution 0.1% RH
Accuracy at 20°C <1%
Long-term stability <1 %rh / year

Training Workshop on Metrology for RAIII & RAIV 24

 Tips & Tricks

• When performing measurements allow enough stabilisation time.

• Result of field verification is a correction table in a narrow
operational range.
• Develop acceptance criteria for instruments.

Training Workshop on Metrology for RAIII & RAIV 25

 Thank you for your attantion

Training Workshop on Metrology for RAIII & RAIV 26