Measurement Uncertainty and

The world leader in serving science Analytical Decision in AS-AES

Edmund Halsz
Thermo Fisher Scientific

ILAP Meeting

Vail Colorado, October 10th 2009

Outlook

Measurement uncertainty: definition, needs for expressing,

possible sources of uncertainty and how to express it

Example of quantifying uncertainty in analytical chemistry

Sources of uncertainty in AS-AES (OES); CRMs

Published examples of expressing uncertainty in AS-AES

Proposal to quantify bias uncertainty in AS-AES

Analytical decision: product conformance

2
Definition

Measurement uncertainty is a parameter associated with the

result of the measurement that characterizes the dispersion of
the values that could be reasonable attributed to the measurand
ISO Guide to the Expression of Uncertainty in Measurements
(GUM)

Measurement uncertainty expresses the doubt about the validity

of any measurement results
The number after

3
Uncertainty and Error

All measurements are affected by errors. The measurement

uncertainty expresses the size that the measurement error might
be

While doubt about validity arises from error, error and uncertainty
are not the same
The doubt may exist about the exactness of the results, i.e., as a
result of repeatability or reproducibility attributable to time period
factors.
The doubt may exist about the validity of a result attributable to
inappropriate sample collection or handling, to heterogeneity leading
to sub-sampling error or to the presence of analytical bias.

4
Parties needing measurement uncertainties

The beneficiary of the results

Uncertainty together with the result to make a correct decision
Important when looking at allowable (legal, normalized)
concentration limit

The testing laboratory

To control and improve quality of measurement
ISO 17025 requires an estimation of the measurement uncertainty

5
General requirements for the competence of testing and
calibration laboratories (ISO/IEC 17025-2005)
Testing laboratories need to estimate the measurements
uncertainty
Procedures to estimate measurement's uncertainty
The degree of necessary rigour in an estimate of the uncertainty of
measurement depends on factors such as:
requirements of the testing method
customer demands
narrow limits on which the decision of conformity to a specification is based

Sources of uncertainty explicitly presented:

reference materials
methods
equipment
environmental conditions
sample preparation method
operator

6
Evaluating Measurement Uncertainty

By measurements and statistical approach

Different sources of uncertainty are estimated from measurements
and combined with statistical formulae into a single value

By using existing knowledge

Basis for the estimation of measurement uncertainty is the existing
knowledge (no special research should be required from the
laboratories). Existing experimental data should be used (quality
control charts, validation, inter-laboratory comparisons, CRMs etc.)
(European Accreditation guidelines)

Guidelines given in GUM, EA guidelines, EURACHEM/CITAC

guide 2, ISO DTS 21748

7
U = k u

Expressing and Using Measurement Uncertainty

Uncertainty is expressed as combined expanded

measurement uncertainty :

U = k u

Coverage factor k=2 provides ~95% level of confidence

8
Potential Sources of Uncertainty in Analytical Measurements

9
Quantifying Uncertainty for Chemical Analysis

Example from NORDTEST Report TR 537: Handbook for calculation

of measurement uncertainty in environmental laboratories
Uncertainty expressed for an analyte at a specified concentration level

Evaluate repeatability (from n control samples and m CRMs): u ( Rw )

Evaluate bias from CRMs n
bias 2 i
RMSbias = i =1
n
m

u (C
j =1
ref )j
u (Cref ) =
1.96 m
u (bias ) = RMSbias + u (Cref ) 2
2

Convert components to standard uncertainty

uc = u ( Rw ) 2 + u (bias ) 2
Calculate combined standard uncertainty
U = 2 uc

10
Some Sources of Uncertainties in AS-AES
1 mm

11s
0s 30s

150s 50s
70s

Profiling and standardization (drift correction)

Measurement of calibration standards
(repeatability)
Certification of reference materials
Calibration model and corrections
Sampling and measurement of unknown sample

11
Reference Materials to Calibrate AS-AES Instruments

Produced, certified and used according to:

ISO Guide 30: Terms and definitions used in connection with reference materials
ISO Guide 31: Contents of certificates of reference materials
ISO Guide 32: Calibration in analytical chemistry and use of certified reference materials
ISO Guide 33: Uses of certified reference materials
ISO Guide 34: Quality system guidelines for the production of reference materials
ISO Guide 35: Certification of reference materials---General and statistical principles
Homogeneity tested by OES and XRF
Well characterized by wet chemistry, AAS, ICP, DCP, ICP-MS, XRF, etc.
Since 1998, uncertainty is specified in the certificates

12
Uncertainty in Certification of Reference Materials

Examples for BAS 402/2, IARM 17B, NIST 1767

13
Uncertainty in Certification of Reference Materials
Example for steels
BAS (Bureau of Analysed Samples Ltd.):
Before 1996: SM
After 1996:ISO Guide 35; u=t*SM/ n
SM: standard deviation of means (averages); inter-laboratory standard deviation
Brammer Standard Company, Inc.:
uncertainties based on value judgments; no attempt to derive from
exact statistical measurements of imprecision
ECISS (European Committee for Iron and Steel)
SM= (sb2+sW2/4)
CKD:
GUM and ISO Guide 35, U (expanded combined uncertainty with
coverage factor 2-2.5)
ARMI (Analytical Reference Materials International)
Before 1996: Standard deviation using n-1 values
After 1996: Confidence interval ~(1/ 2.5)*SM (1/ 3)*SM

14
Uncertainty in Certification of Reference Materials
Example of non-uniform certification uncertainty

15
Expressing Uncertainty for AS-AES

Situation today:
Norms and guides: very few examples
Very few published papers

In published papers bias uncertainty evaluated in a calibration

a posteriori experiment
Measure CRMs (eventually independent ones) and evaluate
RMSbias
Data from calibration never taken into account

16
Bias Uncertainty Evaluation
Calibration a posteriori experiment

T. Drglin- An estimation of measurement uncertainty for an optical

emission spectrometric method, Accred. Qual. Assur. (2003) 8:130-133/
Validation of a low alloy steel calibration procedure:
Based on validation function
Cmeasured = b + a Ccertified
Overall precision estimated as normalized data from duplicate tests on nine
selected CRMs, under repeatability conditions
The overall bias expressed as:

u (bias ) = u (CRM ) 2 + u ( Diff ) 2

s ( Diff )
u ( Diff ) =
n
Note: computation steps not clearly defined

17
Bias Uncertainty Evaluation
Calibration a posteriori experiment
Uncertainty assessment for LAST calibration with ARL 3460
Uncertainty for each element expressed as single value, being a
constant over the complete calibration range

T. Drglin- An estimation of measurement uncertainty for an optical emission spectrometric method,

Accred. Qual. Assur. (2003) 8:130-133/

18
Bias Uncertainty Evaluation
Calibration a posteriori experiment
B. Magnusson- Technical Research Institute of Sweden
OES users seminar, Sigtuna, SE, sept. 2004

Customers requirement: N analysis @ 0.12% with 0.005% in stainless

steel with ES-AES spectrometer
From control diagram (2s): u(Rw)=0.0034%/2=0.0017%
From bias test:

Standard uncertainty: uc=0.00206%

Expanded uncertainty: U=2*uc=0.00412 ~0.004%

19
Building Uncertainty Budgets

From inter-laboratory studies databases:

Horwitz function (William Horwitz IUPAC)
U(method)= 0.303C0.661
Inter-laboratory test data from ISO TC/17 SC 1 (Methods of determination
of chemical composition)
U(method)= 0.384C0.58
Data from 1999-2000 of ASTMs Proficiency Test Programs for Plain
Carbon Low Alloy Steel and Stainless Steel

U(method): 2 x interlaboratory
standard deviation,

C: analyte concentration, m/m, %

20
Uncertainty in AS-AES

For a given method

uncertainty is related to:
Analytical wavelength
Excitation energy
Transition probability
Express bias
Readout channel uncertainty through
Channel noise channels calibration
Excitation source data
Source noise
Sample metallurgical
structure
Homogeneity,
Insoluble phases
Inclusions

21
Proposal for Quantifying Bias Uncertainty in AS-AES

Express bias uncertainty through confidence interval

Example: EURACHEM / CITAC Guide CG4 (2000): Quantifying
Uncertainty in Analytical measurement
Bias uncertainty for linear calibration curves

Explanation:
Confidence intervals capture the true value of the regression
function with a user-specified probability, the confidence level, using
the estimated regression function and the associated estimate of the
error

22
Confidence Interval

Mathematically valid only for linear regression without corrections

Applicable only for linear calibration curves
AA, ICP-AES, ICP-MS, rarely Electrical Spark AES

23
Proposal to Use Confidence Interval to Express Bias Uncertainty

Proposal: calculate confidence interval for the correlation curve

Independent to the calibration mathematical model giving Ccalculated

Ccalculated = a Ccertified + b

1 1 (Cl C ) 2
CI (Cl ) = t ;n k SEE + + n

(C
m n
certified j C )2
j =1
Where:
m: number of replicates for an average
n: number of samples used to build calibration curve
k: total number of coefficients for polynomial and corrections
C : average of all concentrations Ccertified
t: Student factor for probability and n-k degrees of freedom
SEE: Standard Estimate of Error

1 n
SEE = (Ccalculated Ccertified )
2

n k j =1

24
Confidence Interval to Express Bias Uncertainty

25
Confidence Interval to Express Bias Uncertainty

Change the coordinates for correlation curve

Keep the same confidence interval

26
Confidence Interval to Express Bias Uncertainty

27
Properties of Confidence Interval

Hyperbolic shape
Higher uncertainty at the edges
Minimum absolute uncertainty for C
Curve extension modifies / degrades bias uncertainty
Weighted regression improves uncertainty for localized (low) concentrations
Multiple segments per element line produce saw jaws

28
Confidence Interval to Express Bias Uncertainty
Multiple Segments

29
Expressing Combined Uncertainty for AS-AES (proposal)

Measure unknown sample

For each element:
Calculate average concentration Caverage
Calculate repeatability Sd
Calculate bias uncertainty as confidence interval for average concentration
Calculate combined uncertainty

C = Caverage 2u
u = CI (Caverage ) 2 + Sd 2

Note: u does not take into account errors from re-standardization

30
Analytical Decision - Product Conformance

Measurement uncertainty and conformance testing: risk analysis

ASME B89.7.4.1-2005 Technical Report
Applied in work pieces inspections, instrument verifications and general
conformance tests where uncertain numerical test results are compared
with specific requirements
Measurement capability index: Cm = T / 4u = T / 2U
T = TU TL

Probability for conformance:

TU xm ) TL xm )
PC =
u u
Where:
(z): cumulative probability for standard normal distribution
xm: measured value

31
Analytical Decision
Product conformance

Example for Cr and Ni in AISI 304, instrument ARL Quantris

Cr: 18.0-20.0%; uCr~0.15%;
Ni: 8.0-10.5%; uNi~0.07%

32
Analytical Decision
Product conformance

Defining a guard band multiplier

g=h*U

Example for Cr and Ni in AISI 304 for Pc=0.99

gCr=0.35%; stringent acceptance zone: 18.35 - 19.65%
gNi=0.16%; stringent acceptance zone: 8.16 - 10.34%

33