NCSLI Measure 2007 Dec

measure
NCSL INTERNATIONAL
The Journal of Measurement Science Vol. 2 No. 4 December 2007
D
e
c
e
m
b
e
r
2
0
0
7
N
C
S
L
I
n
t
e
r
n
a
t
i
o
n
a
l
m
e
a
s
u
r
e
T
h
e
J
o
u
r
n
a
l
o
f
M
e
a
s
u
r
e
m
e
n
t
S
c
i
e
n
c
e
V
o
l
.
2
N
o
.
4
In This Issue:
Application of Simulation Software
to Coordinate Measurement
Uncertainty Evaluations
NIST Primary Frequency Standards
and the Realization of the SI Second
Leakage Effects in Microwave
Power Measurements
Electromagentic Metrology
Challenges in the U.S. DOD
and the Global War on Terrorism
The new 9140 Series
Field Metrology Wells
deliver maximum
portability, speed,
and accuracy at a
minimum price
The 9140 family from Flukes
Hart Scientic division light-
ens your eld calibration
workload at every turn. Their
compact size and weight
make them the most portable
eld calibrators in their class.
Fast heating and cooling
takes you through your work-
load at twice the speed of
competitive dry wells. And
automated and documented
calibrations save even more
time. Plus the built-in refer-
ence thermometer, two mea-
surement channels, and 24 V
loop power supply ensure top
accuracy without extra tools.
Fluke. Keeping your world
up and running.
To learn more about why

9140 Series Field Metrology Wells
are the best in the eld go to
www.hartscientic.com, or call
1-800-GET-HART (438-4278).
Best in Field.
2007 Fluke Corporation.
All rights reserved. Ad 02133
C l OK_____
L
ICS# 070887 Fluke
9
MEASURE | 1 Vol. 2 No. 4 December 2007
Contents
Features
SPECIAL REPORTS
16 Electromagnetic Metrology Challenges in the U.S. Department
of Defense and the Global War on Terrorism
Larry W. Tarr
22 A Users Guide to the Information in the
BIPM Key Comparison Database
C. Thomas and A. J. Wallard
TECHNICAL PAPERS
28 Calibrating Laser Vacuum Wavelength with a
GPS-based Optical Frequency Comb
Jack Stone, Liang Lu and Patrick Egan
40 Application of Simulation Software to Coordinate
Measurement Uncertainty Evaluations
Jon M. Baldwin, Kim D. Summerhays,
Daniel A. Campbell and Richard P. Henke
54 Leakage Effects in Microwave Power Measurements
Ronald A. Ginley, Denis X. LeGolvan and Ann F. Monke
60 A Comparison in Conventional Mass Measurements
Between SENCAMER (Venezuela) and CESMEC-LCPN-M (Chile)
Francisco Garcia, Elias Salazar, Ral Hernndez,
Luis Rojas, Haygas Kalustian and Fernando Leyton
REVIEW PAPERS
66 Overview of ASME B89 Standards with an Emphasis on
B89.4.22Methods for Performance Evaluation of
Articulated Arm Coordinate Measuring Machines
Brian Parry
74 NIST Primary Frequency Standards
Michael A. Lombardi, Thomas P. Heavner and Steven R. Jefferts
Departments
3 Letter from the Editor
5 International NMI News
9 Metrology News
91 New Products
96 Advertiser Index
measure
NCSL INTERNATIONAL
Welcome to NCSLI measure,
a metrology journal published by
NCSL International (NCSLI), for the
benefit of its membership.
CONTACT NCSLI
Business Office:
Craig Gulka, Business Manager
NCSL International
2995 Wilderness Place, Suite 107
Boulder, CO 80301-5404 USA
Phone: 303-440-3339
Fax: 303-440-3384
Email: info@ncsli.org
NCSLI measure Information:
www.ncsli.org/measure/
see page 15
2 | MEASURE www.ncsli.org
Members Only
Over a decade of archived technical/managerial
articles for today's metrologist
NCSLI Technical Publications Library
NCSLI Newsletter collection is available for articles
and information
Stay current with committee activities
Member Delegates can
register their entire staff,
allowing individual access.
NCSLI measure (ISSN #19315775) is a metrology journal published by
NCSL International (NCSLI). The journal's primary audience is calibra-
tion laboratory personnel, from laboratory managers to project leaders
to technicians. measure provides NCSLI members with practical and
up-to-date information on calibration techniques, uncertainty analysis,
measurement standards, laboratory accreditation, and quality
processes, as well as providing timely metrology review articles. Each
issue will contain technically reviewed metrology articles, new prod-
ucts/services from NCSLI member organizations, technical tips,
national metrology institute news, and other metrology information.
Information for potential authors, including paper format, copyright
form, and a description of the review process is available at
www.ncsli.org/measure/ami.cfm. Information on contributing Technical
Tips, new product/service submission, and letters to the editor is avail-
able at www.ncsli.org/measure/tc.cfm. Advertising information is avail-
able at www.ncsli.org/measure/ads.cfm.
Managing Editor
Richard B. Pettit, Sandia National Laboratories (Retired), 7808 Hendrix,
NE, Albuquerque, NM 87110 USA. Email: randepettit@comcast.net
NMI/Metrology News Editor:
Michael Lombardi, NIST, Mailcode 847.00, 325 Broadway, Boulder, CO
80305-3328 USA. Email: lombardi@nist.gov
New Product/Service Announcements:
NCSLI Business Office, 2995 Wilderness Place, Suite 107, Boulder, CO
80301-5404 USA. Email: lstone@ncsli.org
Technical Support Team:
Norman Belecki, Retired, 7413 Mill Run Dr., Derwood, MD
20855-1156.
Belinda Collins, National Institute of Standards and Technology
(NIST), USA
Salvador Echeverria, Centro Nacional de Metrologia (CENAM), Mexico
Andy Henson, National Physical Laboratory (NPL), United Kingdom
Klaus Jaeger, Jaeger Enterprises, USA
Dianne Lalla-Rodrigues, Antigua and Barbuda Bureau of Standards,
Antigua and Barbuda
Angela Samuel, National Measurement Institute (NMI), Australia
Klaus-Deter Sommer, Physikalisch-Technische Bundesanstalt (PTB),
Germany
Alan Steele, National Research Council (NRC), Canada
Pete Unger, American Association for Laboratory Accreditation (A2LA),
USA
Andrew Wallard, Bureau International des Poids et Mesures (BIPM),
France
Tom Wunsch, Sandia National Laboratories (SNL), USA
Production Editor:
Mary Sweet, Sweet Design, Boulder, CO 80304 USA
Email: msweet@sweetdesign.com
Copyright 2007, NCSL International. Permission to quote excerpts or to reprint
any figures or tables from articles (Special Reports, Technical Papers, Review
Papers, or Technical Tips) should be obtained directly from an author. NCSL Inter-
national, for its part, hereby grants permission to quote excerpts and reprint
figures and/or tables from articles in this journal with acknowledgment of the
source. Individual teachers, students, researchers, and libraries in nonprofit institu-
tions and acting for them are permitted to make hard copies of articles for use in
teaching or research, provided such copies are not sold. Copying of articles for
sale by document delivery services or suppliers, or beyond the free copying
allowed above, is not permitted. Reproduction in a reprint collection, or for adver-
tising or promotional purposes, or republication in any form requires permission
from one of the authors and written permission from NCSL International.
measure
NCSL INTERNATIONAL
The Journal of Measurement Science
www.MyNCSLI.org
On-line resources for your organization
Letter from the Editor
HOW TO REACH US: MAIL letters to: NCSLI measure Journal, 2995 Wilderness Pl., Ste 107, Boulder, CO 80301-5404 USA
FAX letters to: 303-440-3384 E-MAIL letters to: editor@ncsli.org
This issue marks the end of the second year publishing NCSLI measure. Over these two years, a total of 67
technical papers have been published, including ten Special Reports, eight Review Papers, and four Technical
Tips. As you may have noticed, we increased the journal size in 2007 to 96 pages, from 80 pages last year. As
always, I continue to receive wonderful support for each issue, including Mary Sweet and her staff who
convert each paper to electronic, publication form and then pull everything together, always on time and very
professionally done; Michael Lombardi, NIST, who collects and edits all the NMI News and Metrology News
items; Linda Stone, NCSLI, who collects and edits all the new product/service announcements; and Craig
Gulka, NCSLI, who always provides valuable ideas and suggestions.
I was recently reminded that October 4, 2007 was the 50th anniversary of the launching of the Sputnik
satellite by Russia. The satellite was an amazing 83 kilos (183 pounds) and broadcast a beeping radio signal
as it circled the earth. From a scientific viewpoint, Sputnik carried no research instruments but the gradual
changes in Sputniks orbit due to atmospheric drag forces allowed researchers to reconstruct the atmospheric
density at the satellites altitude. These observations lead to an increased investigation of the Earths outer
atmosphere. Sputnik sent shock waves of concern within the United States about the decline in scientific
leadership, so Congress initiated a science and math education push (National Defense Education Act). The
next year, 1958, Congress created the National Aeronautics and Space Administration (NASA), which started
the Apollo Program in 1961; as we all remember, this resulted in the landing (and returning) a man on the
moon July 20, 1969. I remember those events and was excited to be studying math and physics as I worked
my way through school. For several summers, I had a job working with scientists at NASAs Lewis Research
Center (now the Glenn Research Center) analyzing experimental data on the flow of liquid hydrogen, a
possible rocket fuel.
It was also 40 years ago that the unit of time, the second, was redefined in the International System of
Units (SI) so that it was based on the cesium atom. Prior to 1967, the second was based on astronomical time
scales. In this issue of measure, Michael Lombardi, Thomas Heavner, and Steven Jefferts of the National
Institute of Standards and Technology, have authored a review article on the 50-year history of NIST primary
frequency standards that realize the SI second. Besides the interesting photos and discussion, the paper
presents a plot of the uncertainty assigned to nine different NIST cesium primary frequency standards
covering the period from 1950 to today. The results show that there has been an improvement in the
uncertainty of the NIST primary frequency standards of a factor of 10 for every decade in time! This has
resulted in an uncertainty estimate for the new NIST-F2 standard that is under development of less than
1 10
16
(k = 1). These rapid developments are certainly one of the most incredible achievements by
humans in the science of measurements!
There are several other metrology articles that should be of interest to NCSLI members. The article titled
Electromagnetic Metrology Challenges in the U.S. DOD and the Global War on Terrorism, by Larry Tarr,
U.S. Army Primary Standards Laboratory, discusses new developments in the RF and millimeter-wave
portions of the electromagnetic spectrum (up to 100 GHz) that present challenges to the metrology
community. These challenges include new imaging systems that can reveal concealed weapons, mines, and
explosives; radio frequency identification systems for identifying and tracking assets; antenna parameter
metrology for gain, pattern, and polarization data; development of synthetic instrumentation through the use
of software; and complex on-chip testing at RF and millimeter wave frequencies. Advances in these areas will
require coordination between government agencies, national metrology institutes, manufacturers, and
metrology organization, such as NCSLI.
Another paper by Brian Parry, Boeing, won a Best Paper Award at the 2007 NCSLI Conference is entitled
Overview of ASME B89 Standards with an Emphasis on B89.4.22Methods for Performance Evaluation
of Articulated Arm Coordinate Measuring Machines. This standard addresses evaluation of articulated
arm coordinate machines by specifying a minimum set of requirements that can be used to determine
machine performance.
Richard Pettit
Managing Editor
Sandia National Laboratories (Retired)
NMI NEWS
PTB Tests Quantum Voltmeter for
Alternating Voltages
The quantum voltmeter for alternating voltages conceived at
the Physikalisch-Technische Bundesanstalt (PTB) has achieved
in its test phase an uncertainty of 5 10
8
during the measure-
ment of a 400 Hz signal, a value ten times lower than previously
obtained. The outstanding performance of superconducting
quantum standards, used so far for dc voltage calibrations, has
thus been extended for the measurement of alternating voltages.
In the low frequency range, alternating voltages are measured
using sampling methods during which the time-varying voltage
is measured repeatedly (sampled) in rapid succession. The
amplification factor and the internal voltage reference of the
sampling voltmeter limits the attainable uncertainty. In practice,
this can be completely avoided if the sampled voltage is directly
compared with the voltage of a Josephson quantum standard,
which is known at 1 V to better than 0.1 nV.
This idea is realized in a method developed and patented at
PTB. For this purpose, alternating voltages are synthesized with
programmable Josephson-series-arrays. A chip, cooled down to
the temperature of liquid helium, contains 8 192 superconduct-
ing Josephson tunneling junctions supplied with a microwave
frequency of 70 GHz. They are distributed over segments with
1, 2, 4, 8, 16, junctions. Switchable current sources control
the individual segments such that they produce quantized
partial voltages which add up to the total voltage. A transition
between quantized voltages requires less than 100 ns; thus, the
slowly changing voltage to be measured can be compensated. If
the two time-varying voltages and a sampling voltmeter are syn-
chronized, the differences between the two alternating voltages
can be measured with high resolution.
It is now possible, with new programmable Josephson cir-
cuits currently only produced at PTB, to synthesize alternating
voltages with amplitudes of even 10 V, making possible a range
of additional applications. In addition, the attainable relative
measurement uncertainty should, due to the improved signal-
to-noise ratio, decrease by an additional order of magnitude.
For more information, contact R. Behr,
email: ralf.behr@ptb.de
PTB Utilizes Robot Goniophotometers for
Measurement of the Luminous Flux
A worldwide unique goniophotometer with three long-armed
robots has now been put into operation at the Physikalisch-
Technische Bundesanstalt (PTB). With its newly developed pho-
tometer heads, the goniophotometer can simultaneously detect
photometric, radiometric and colorimetric quantities, as well as
relative spectral distributions, by means of a charged-coupled
device (CCD) array spectrometer.
PTB has the task of distributing to industry the lumen unit
of the luminous flux; which is fundamentally derived from the
SI base unit candela for the luminous intensity. To calibrate
transfer standards used for this purpose, a goniophotometer in
a fully gimballed suspension construction was developed in the
1970s and used at PTB. Metrological limitations and an out-
dated computer technology led to the need to redesign the
system, making it more modern and at the same time reducing
the measurement uncertainties.
The new goniophotometer of completely novel design, whose
concept is protected by an international patent, was developed
at PTB and has now been put into operation. It is composed of
three robots, each having seven controlled axes for moving the
slim arms having a length of more than 6.4 m. One robot
carries the light source in a freely selectable burning position,
aligns it in the instrument center and holds it in position during
the measurement. The other two robots each align a photome-
ter head with the light source and divide the room into hemi-
spheres. They can move on any paths at distances of 1 m to 3 m
and with measurement periods of typically 10 min to 1 h. The
orientation of the robots in the room, as well as their kinematic
characterization, is determined by means of a laser tracker
system. This results in path deviations of the photometer head
of < 0.6 mm, and on average only 0.2 mm. The movable pho-
tometer heads and a monitor-photometer head are each
designed as a tristimulus colorimeter head with four channels.
In addition, they contain an unfiltered Si-photodiode for radio-
metric measurements and a CCD array spectrometer. Thus,
light, color and optical radiation are measured through the
same light-entry window, all photo currents are measured in
parallel and converted to frequencies. This allows the simulta-
neous measurement of all 18 channels with synchronous trig-
gering and any integration times whatsoever, optimally adapted
to the motion sequences of the robots and the modulations of
NMI NEWS
Part of a Josephson-series-array. From the left, the microwave
striplines appear; from below the control leads for the individual seg-
ments.
Continued on page 6
If you would like your news item to appear in a future issue
of measure, contact Michael Lombardi at lombardi@nist.gov.
NMI NEWS
the light.
With the new robot goniophotometers, it is now possible to
determine both the photometric and chromaticity data of light
sources at considerably reduced measurement periods. By using
the freely programmable robots, nearly any near-field and far-
field measurements are possible. The ozone-proof version of the
goniometer also allows the measurement of UV lamps, and, due
to the adjustable temperature range of 25 C to 35 C, lumines-
cent lamps can also be measured at the respective luminaire
temperature.
For more information, contact M. Lindemann, email:
matthias.lindemann@ptb.de
NIST Light Source Illuminates Fusion
Power Diagnostics
Using a device that can turn a tiny piece of laboratory space into
an ion cloud as hot as those found in a nuclear fusion reactor,
physicists at the National Institute of Standards and Technology
(NIST) are helping to develop one of the most exotic yard-
sticks on earth, an instrument to monitor conditions in the
plasma of an experimental fusion reactor. Their measurement
tool also is used in incandescent light bulbs, the element tungsten.
The intended beneficiary of this research is ITER
(www.iter.org), a multinational project to build the worlds
most advanced fusion test reactor. ITER, now under construc-
tion in Cadarache, France, will operate at high power in near-
steady-state conditions, incorporate essential fusion energy
technologies and demonstrate safe operation of a fusion power
system. It will be a Tokamak machine, in which a hot
250 000 000 C plasma of hydrogen isotope ions, magneti-
cally confined in a huge toroidal shape, will fuse to form helium
nuclei and generate considerable amounts of energy, much the
same way energy is generated in the sun.
One major issue is how to measure accurately the tempera-
ture and density of the plasma, both of which must reach crit-
ical values to maintain the fusion process. Any conventional
instrument would be incinerated almost instantly. The usual
solution would be to use spectroscopy: monitor the amount and
wavelengths of light emitted by the process to deduce the state
of the plasma. But light comes from electrons as they change
their energies, and at Tokamak temperatures the hydrogen and
helium nuclei are completely ionized no electrons left. The
answer is to look at a heavier element, one not completely
ionized at 250 000 000 degrees, and the handy one is tungsten.
The metal with the highest melting point, tungsten is used for
critical structures in the walls of the tokamak torus, so some
tungsten atoms always are present in the plasma.
To gather accurate data on the spectrum of highly ionized
tungsten, as it would be in the Tokamak, NIST physicists use an
electron beam ion trap (EBIT), a laboratory instrument which
uses a tightly focused electron beam to create, trap and probe
highly charged ions. An ion sample in the EBIT is tiny a
glowing thread about the width of a human hair and two to
three centimeters longbut within that area the EBIT can
produce particle collisions with similar energies to those that
occur in a fusion plasma or a star. In a pair of papers,
1
the NIST
researchers uncovered previously unrecognized features of the
tungsten spectrum, effects only seen at the extreme tempera-
tures that produce highly charged ions. The NIST team has
reported several previously unknown spectral lines for tungsten
atoms with 39 to 47 of their 74 electrons removed. One partic-
ularly significant discovery was that an anomalously strong
spectral line that appears at about the energies of an ITER
Tokamak is in fact a superposition of two different lines that
result from electron interactions that, under more conventional
1
Yu. Ralchenko, Density dependence of the forbidden lines in Ni-like
tungsten, J. Phys. B: At. Mol. Opt. Phys., vol. 40, pp. F175-F180,
2007.
Yu. Ralchenko, J. Reader, J.M. Pomeroy, J.N. Tan and J.D. Gillaspy,
Spectra of W(39+)-W(47+) in the 12-20 nm region observed with
an EBIT light source, J. Phys. B: At. Mol. Opt.Phys., vol. 40, pp.
3861-3875, 2007.
A tiny human figure indicates the scale of the ITER toroid
View inside the robot goniophotometer with an LED array as light
source in the equipment centre and the two measuring robots in basic
position. The group of people provides a size comparison.
NMI NEWS
plasma conditions, are too insignificant to show up.
Team member John Gillaspy observes: Thats part of the fas-
cination of these highly charged ions. Things become very
strange and bizarre. Things that are normally weak become
amplified, and some of the rules of thumb and scaling laws that
you learned in graduate school break down when you get into
this regime. The team has proposed a possible new fusion
plasma diagnostic based on their measurements of the superim-
posed lines and supporting theoretical and computational
analyses.
New NIST Calibration Service Supports
Power Grids
The new calibration service for phasor measurement units
(PMUs) offered by the National Institute of Standards and
Technology (NIST) benefits the operators of Americas electri-
cal power grid, as well as everyone who values
uninterrupted electrical power. The service provides calibra-
tions for the instruments that measure the magnitude and phase
of voltage and current signals in a electrical power system, a
combined mathematical entity called a phasor, and report the
data in terms of Coordinated Universal Time (UTC, also known
as the official world atomic time). Use of absolute time
enables measurements called phase angles taken at one location
on a power grid to be comparable to others across different
systems. Phase angles and their derivations allow grid managers
to know the operating condition of their portion of the system
and determine if action is needed to prevent a power blackout.
The new NIST calibration service has already yielded two
additional benefits. First, a major PMU manufacturer reports
that using the calibrations during the manufacture of its instru-
ments has improved their accuracy by a factor of five. Secondly,
some PMUs that have been calibrated using the NIST service
have revealed incompatibilities in the message format they send
out, leading to corrections that have improved interoperability
between PMUs across power grids.
This project is partially funded by the U.S. Department of
Energy (DOE), and is operated in conjunction with DOE and
the North American Synchrophasor Initiative (NASPI). NASPI
is a joint government and utility collaboration supporting the
North American Electric Reliability Corporations efforts to
improve the reliability of the nations power grids.
For more information on the NIST PMU calibration service,
contact Jerry Stenbakken, gerard.stenbakken@nist.gov,
(301) 975-2440.
More News on page 9
8uIomaIed caIihraIion soIImare
miIh a puIse.
You probabl y know SureCAL
for i ts l eadi ng cal i brati on software products that hel p you

maintain your instruments to exacting precision. But we also have a human side, whose hallmark
is unparalleled customer service. We believe in the value of an intelligent voice, so we feature
live customer support. Its one more way SureCAL
provides high technology with a human touch.
2
0
0
6
N
o
r
t
h
r
o
p
G
r
u
m
m
a
n
C
o
r
p
o
r
a
t
i
o
n
www.northropgrumman.com Search: SureCAL
P
METROLOGY NEWS
EURAMET Calibration Guides Updated
Ten EURAMET (European Associ-
ation of National Metrology Insti-
tutes) calibration guides (formerly
European Co-operation for Accredi-
tation, EA, guides) have been recently updated. They are all
available for downloading from the EURAMET web site at:
www.euromet.org/docs/calguides/
The EURAMET documents cover various temperature cali-
brations, inductive voltage dividers, vector network
analyzers, static torque measuring devices, digital multimeters,
hardness measurements, electromechanical manometers, and
non-automatic weighing instruments. A summary of each cali-
bration guide is provided below:
1. Calibration of Thermocouples, EURAMET/cg-08/v.01, July
2007. This guide serves as a basic advisory document for
laboratories that calibrate thermocouples. It is primarily for
thermocouple types standardized in accordance with tem-
perature-emf reference tables produced at NIST and
adopted by the International Electrotechnical Commission
(IEC) and later by the European Committee for Standardi-
zation (CEN) as EN 60584-1:1996. It covers the tempera-
ture range -200 C to +1600 C, the calibrations being
carried out in terms of the International Temperature Scale
of 1990 (ITS-90). Most of the topics covered in this guide
are also applicable to non-standard thermocouples.
2. Measurement and Generation of Small AC Voltages with
Inductive Voltage Dividers, EURAMET/cg-09/v.01, July
2007. This guide applies to the generation and measure-
ment of small AC voltages from 1 mV to 1 V in the fre-
quency range from 50 Hz to 100 kHz depending on the
selected procedure and the measuring method used. The
accreditation of the measurand AC voltage for voltages of
more than 1 V is presupposed.
3. Guidelines on the Calibration of Temperature Indicators
and Simulators by Electrical Simulation and Measurement,
EURAMET/cg-11/v.01, July 2007. This guide applies to the
calibration, by electrical simulation and measurement, of
temperature indicators and temperature simulators intended
for use with resistance thermometers or standardized ther-
mocouples. It also applies to the calibration of simulators
which are intended to emulate the electrical outputs of
resistance thermometers or standardized thermocouples.
4. Guidelines on the Evaluation of Vector Network Analyzers
(VNA), EURAMET/cg-12/v.01, July 2007. This guide
describes how to evaluate some of the important character-
istics of VNAs. It describes measurement procedures that
can be used to assess whether or not a VNA meets the
accreditation requirements of EN45001. The principles
given in this guide apply to any frequency range for which
VNAs can be used, and to any transmission medium;
however, some of the techniques given for the assessment of
uncertainties are only applicable to coaxial lines at frequen-
cies above 500 MHz. Although the calibration of a VNA
must, by definition, cover phase as well as magnitude capa-
bilities, the uncertainties produced using this document are
only applicable to magnitude. Phase uncertainty will be
covered in a future edition.
5. Calibration of Temperature Block Calibrators,
EURAMET/cg-13/v.01, July 2007. This guide applies to
temperature block calibrators where a controllable temper-
ature is realized in a solid-state block, with the aim of cali-
brating thermometers in the borings of this block. A
temperature block calibrator comprises at least the solid-
state block, a temperature-regulating device for the block,
and a temperature sensor with an indicator (the built-in con-
trolling thermometer) to determine the block temperature.
These components are either combined to form a compact
unit, or kept separate. This guide is valid in the temperature
range from 80 C to +1300 C, but the temperature ranges
stated by the manufacturer shall not be exceeded.
6. Guidelines on the Calibration of Static Torque Measuring
Devices, EURAMET/cg-14/v.01, July 2007. This guide
applies to torque measuring devices where the torque is
obtained by the measurement of the elastic deformation of
a body or of a measurand proportional to it. This guide
applies to the static calibration of torque measuring systems
using supported beams or the comparison method with ref-
erence transducer and includes an example for calculation of
the uncertainty of measurement. A diagram showing an
example of the calibration steps and series is given in Annex
D. The guide defines the torque measuring device s the com-
plete instrument comprising all parts, from the torque trans-
ducer to the indicating device.
7. Guidelines on the Calibration of Digital Multimeters,
EURAMET/cg-15/v.01, July 2007. This document provides
guidelines on the calibration of Digital Multimeters (DMM)
for accredited calibration laboratories (ACL). In the absence
of specific international written standards on DMMs, this
document supplements the manufacturers recommenda-
tions and the calibration procedures of the ACLs. Even
though this guide is not intended to cover the question of
judging whether or not a DMM is compliant with a specifi-
cation, it suggests a suitable calibration method on which a
statement of compliance can be based.
8. Guidelines on the Estimation of Uncertainty in Hardness
Measurements, EURAMET/cg-16/v.01, July 2007. In the
field of hardness measurement a wide variety of methods
and equipment is applied which may differ according to the
material. A hardness measurement is useful when the results
obtained at different sites are compatible to within a deter-
mined interval of measurement uncertainty. This guide
demonstrates and applies the concepts of measurement
uncertainty to this special field.
9. Guidelines on the Calibration of Electromechanical
Manometers, EURAMET/cg-17/v.01, July 2007. Scope:
This guide discusses the calibration of electromechanical
manometers, excluding dial gauges. It provides the users of
METROLOGY NEWS
Continued on page 10
METROLOGY NEWS
electromechanical manometers with the fundamentals nec-
essary for establishing and applying calibration procedures.
The guide applies to all electromechanical manometers for
measuring absolute, gauge or differential pressures, exclud-
ing vacuum devices measuring pressures below 1 KPa.
10. Guidelines on the Calibration of Non-Automatic Weighing
Instruments, EURAMET/cg-18/v.01, July 2007. This docu-
ment provides guidance for the static calibration of self-indi-
cating, non-automatic weighing instruments. It covers the
measurements to be performed; the calculation of the meas-
uring results; the determination of the uncertainty of meas-
urement; and the contents of calibration certificates. The
object of the calibration is the indication provided by the
instrument in response to an applied load. The results are
expressed in units of mass. This guide does not specify lower
or upper boundaries for the uncertainty of measurement.
The calibration provider and customer must agree on an
appropriate level of uncertainty of measurement; based on
the use of the instrument and the cost of the calibration.
New EUROLAB Documents on Uncertainty
and ISO 17025 Computer Guidance
EUROLAB (European Federation of
National Associations of Measure-
ment, Testing and Analytical Labora-
tories) has recently published several documents dealing with
the uncertainty of measurement and ISO/IEC 17025 guidance
for computers and software. These documents are available on
the EUROLAB web site (www.eurolab.org/pub/i_pub.html)
and are described below:
1. Measurement Uncertainty Revisited: Alternative Approaches
to Uncertainty Evaluation, EUROLAB Technical Report #
1/2007, March 2007. This report focuses on reviewing and
comparing the currently available approaches for evaluating
measurement uncertainty of quantitative test results, provid-
ing a range of examples. After more than ten years, the ISO
Guide to the Expression of Uncertainty in Measurement,
known as the GUM, is acknowledged as the master docu-
ment on measurement uncertainty throughout the testing
community. This report applies the term measurement
uncertainty to all types of quantitative test results, and the
GUM principles are fully accepted.
2. Guide to the Evaluation of Measurement Uncertainty for
Quantitative Test Results, EUROLAB Technical Report
# 1/2006, August 2006. This document offers technical
guidance on the evaluation of measurement uncertainty for
quantitative test results. While fully compliant with the prin-
ciples of the GUM, this document also includes alternative
approaches to the bottom-up approach, based on a com-
prehensive mathematical model of the measurement
process, as emphasized in the GUM. These alternative top-
down approaches utilize performance data from inter-lab-
oratory comparisons and from within-laboratory validation
and quality control data. Various annexes in the document
provide information about frequently occurring uncertainty
sources and data evaluation problems.
3. Guidance for the Management of Computers and Software
in Laboratories with Reference to ISO/IEC 17025/2005,
EUROLAB Technical Report # 2/2006, October 2006. This
succinct and comprehensive guideline focuses on managing
the requirements specific for computers and software with
respect to ISO/IEC 17025 requirements. This document
identifies neither best practice nor a total solution, but pro-
vides advice and guidance with no mandatory parts. It is
assumed that a laboratory will have measures to comply
with the general requirements of ISO 17025. Therefore this
guidance focuses on the special requirements concerning
software and computer system validation, including: identi-
fication and interpretation of computer and software
clauses in ISO 17025; implementing computing systems in
the lab.; different categories of software; risk assessment;
verification and validation of software; electronic docu-
ments handling, transmission and archiving; usage of com-
puter networks in connection with the measurement
process; and security.
ILAC Celebrates 30 years (1977-2007)
ILAC (International Laboratory Accreditation Cooperation) is
the international cooperation of laboratory and inspection
accreditation bodies which this year celebrates its 30th
anniversary. One of the primary aims of ILAC is the removal of
technical barriers to trade.
Imagine you are importing toys
from another country. The toys
have been tested in that country
by a laboratory which says it
meets international safety stan-
dards. But how do you convince
your authorities that the tests
and results are genuine? That
dilemma was a major impediment
to international trade 30 years ago, but
today, thanks to a handful of pioneers, things have changed for
the better.
The ability of authorities to trust technical standards and pro-
cedures from different countries reaches an important mile-
stone this year, with ILAC celebrating 30 years of helping the
worlds economies overcome technical barriers to trade.
ILACs evolution was prompted by the Tokyo round of inter-
national trade negotiations under the General Agreement on
Tariffs and Trade (GATT). The outcome was the GATT Stan-
dards Code, an agreement between a number of the member
states encouraging recognition of the equivalence of different
standards, and the variety of testing and accreditation regimes.
ILACs Chair, Daniel Pierre said: The first conference on
International Laboratory Accreditation, was convened in
Copenhagen in 1977 by Mr. Per Lund Thoft of the Ministry of
Trade, Denmark with the support of Dr. Howard Forman of the
US Department of Commerce. Twenty countries from around
the world, the EEC Commission and ISO accepted their invita-
tion.
The conference gave countries that already had, or were plan-
METROLOGY NEWS
ning accreditation schemes an opportunity to compare notes
and experiences. The delegations with practical experience were
The National Testing Board of Denmark (1977), NVLAP U.S.
Department of Commerce (1976), BSIs System of the Registra-
tion of Test House, UK (1977), NATA Australia (1947), and
TELARC - New Zealand (1973).
The outcome of that first ILAC conference was the idea that
mutual recognition agreements between accreditation bodies
meant any laboratory, anywhere could have their test results
recognized as reliable. ILAC from its inception has worked to
create an international framework to support international
trade through the removal of technical barriers. This is now rec-
ognized through the ILAC Mutual Recognition Arrangement
(MRA).
Fifty eight signatories, representing 46 economies have now
signed the ILAC Mutual Recognition Arrangement, enhancing
the acceptance of products and services across national
borders.
Further information about ILAC is available from:
www.ilac.org/aboutilac.html
ISO 9001 Celebrates Its 20th Birthday
This year, 2007, marks the twentieth anniversary of ISO 9001,
the International Standard for quality management systems
from the International Organization for Standardization (ISO).
ISO 9001:2000 is the principal standard in the ISO 9000
family, a suite of sixteen standards
that provide guidance on topics such
as performance improvement, audit-
ing and training. Now used in more
than 160 countries around the globe,
ISO 9001 has become an interna-
tional reference for quality manage-
ment requirements, and a benchmark
for improving customer satisfaction and achieving continual
improvement of an organizations performance in pursuit of
these objectives. ISO 9001 can be applied to any organization,
in any sector of activity, whether it is a business enterprise,
public administration, or government department.
The ISO 9000 standards originated as quality assurance stan-
dards in the military, nuclear and construction fields. They grew
through civilian use into national standards before being sub-
mitted to ISO in 1979. Today, there are more than 800,000
ISO 9001 certifications worldwide.
New EURACHEM-CITAC
Uncertainty Guides 2007
A new (2007) EURACHEM-CITAC Guide titled Use of Uncer-
tainty Information in Compliance Assessment has been pub-
lished. The guide applies to compliance with regulatory or
manufacturing limits, where a decision is made on the basis of
a measurement result and its associated uncertainty. The guide
covers cases where the uncertainty does not depend on the
value of the measurand, and cases where the uncertainty is pro-
portional to the value of the measurand. The guide assumes
that the uncertainty has been evaluated by an appropriate
method that takes all relevant contributions into account.
A second new (2007) EURACHEM-CITAC Guide titled
Measurement Uncertainty Arising from Sampling: A Guide to
Methods and Approaches, has also been published. The guide
describes two main approaches to the estimation of uncertainty
from sampling: (1) The Empirical Approach uses repeated sam-
pling and analysis under various conditions, (2) The Modeling
Approach uses a predefined model that identifies each of the
component parts of the uncertainty, makes estimates of each
component, and combines them into an overall estimate. Exam-
ples from both approaches are given, covering a wide range of
different application areas.
For a PDF copy of the new EURACHEM-CITAC guides,
visit: www.eurachem.org
Jeff Gust joins Bagan, Inc.
as VP of Metrology Services
Former NCSLI President Jeff Gust joined Bagan, Inc. in Sep-
tember 2007, as the Vice President of Metrology Services. Gust
brings his extensive knowledge of measurement science and cal-
ibration quality system requirements to Bagan, and will
manage their in-lab and on-site metrology operations.
Jeff Gust has a bachelors degree in Physics from Purdue Uni-
versity and over twenty years of experience in the metrology
industry. He began his career in 1985 as a TMDE repair tech-
nician in the United States Marine Corps. Upon leaving the
Marines, he was employed by Tektronix as a Calibration Tech-
nician and Quality Manager for the Irvine CA Tektronix service
facility. In 1990, Jeff joined Verizon and served as a Metrology
Technician until 1995, when he was promoted to Staff Engineer
and Technical Manager for Verizons Fort Wayne IN metrology
laboratory. There he developed Verizons corporate metrology
quality system and numerous calibration processes. He then
became the VP and Director of Quametec Proficiency Testing
Services, where he developed their quality system and estab-
lished A2LA accredited measurement proficiency testing pro-
grams to support the metrology industry. Gust served as
President of NCSL International in 2006 and continues to serve
on the Board of Directors. He is also a lead assessor for the
A2LA Calibration Accreditation Program, a NACLA evaluator,
and the author of a number of metrology related publications.
Bagan is A2LA accredited to ISO/IEC 17025:2005 for in-lab
and on-site calibration and specializes in calibration programs
for automotive, aerospace, pharmaceutical, and manufacturing
industries, as well as testing labs, and military installations.
For more information, contact Lydia McDevitt,
lmcdevit@rjbagan.com, or visit: www.rjbagan.com
More News on page 12
METROLOGY NEWS
EUROLAB and JRC-IRMM Sign
Cooperation Agreement
Cooperation among testing laboratories is key for the harmo-
nization of test methods and the mutual acceptance of test
results. Optimizing the use of resources is also one of the corner
stones of the strategy of European Federation of National Asso-
ciations of Measurement, Testing and Analytical Laboratories
(EUROLAB) and the Joint Research Centre (JRC) of the Euro-
pean Commission. The representatives of the two organizations
signed a Memorandum of Understanding (MoU) on September
11, 2007 at the JRC Institute for Reference Materials and Meas-
urements (IRMM), in Geel, Belgium. The MoU establishes the
principles and guidelines of their cooperation. The two organ-
izations intend to:
Ensure information exchange in relevant matters, e.g.,
between the JRCs Community Reference Laboratories and
the EUROLAB community.
Coordinate workshops and seminars of mutual interest,
Exchange views and, wherever possible, find consensus posi-
tions on policy matters of interest to the European trade and
innovation community, such as prestandardization research
activities or quality assurance standards.
Coordinate European and international participation for the
standardization and conformity assessment of measurements.
The MoU was signed by the President of EUROLAB, Jean-
Luc Laurent, and the Director of JRC-IRMM, Alejandro
Herrero. Both Mr. Laurent and Mr. Herrero stressed that this
cooperation strengthens the roles EUROLAB and IRMM play
in harmonizing the European measurement system. The
concept of mutual recognition for instance, is very important
for the society as it has economical implications in trade, said
Mr. Herrero.
The President of EUROLAB, Jean-Luc Laurent, and the Director of JRC-
IRMM, Alejandro Herrero, sign the Memorandum of Understanding.
The new Fluke 9640A
RF Reference Source
helps you cut RF
calibration time in half.
2005 Fluke Corporation. All rights reserved. Ad Number 01984.
With the Fluke 9640A RF Reference Source, you can accurately calibrate
a broad RF measurement workload in half the time of traditional RF
calibration solutions. The 9640A gives you a unique combination of
level accuracy, dynamic range, and frequency in a single box with:
Built-in signal leveling and
attenuation.
Level accuracy up to 0.05 dB
from 10 Hz to 4 GHz, which
gives you performance and
precision equal toor better
thanthe HP 3335A over a
wider frequency range.
Easy integration with
MET/CAL
Plus Calibration
Management Software for
higher throughput can reduce
cal time even more.
See the Fluke 9640A at
www.uke.com/9640A.
AM & FM Modulation Sweep frequency Leveled sine
Fluke. Keeping your world up and running.
METROLOGY NEWS
For more information contact: Jean-Marc Aublant, email:
jean-marc.aublant@lne.fr, www.eurolab.org or Doris Florian,
email: doris.florian@ec.europa.eu, www.irmm.jrc.be,
www.jrc.ec.europa.eu
12th IMEKO Symposium
on Man, Science & Measurement
The International Measurement
Confederation (IMEKO) Joint
Symposium on Man, Science
& Measurement will explore emerging challenges and novel
concepts in Measurement Science from an anthropocentric
viewpoint and will highlight education and training in these
areas. The Symposium, which is sponsored by IMEKO commit-
tees TC1 (Education and Training in Measurement and Instru-
mentation) and TC7 (Measurement Science), will be held in
Annecy, France, September 3-5, 2008, and is being organized
by the University of Savoie.
The Symposium will focus on the role of Measurement
Science in human activity. The official language of the confer-
ence is English. Topics will include:
Measuring systems as a way for helping human perception of
the world
Measurement as a way for helping perception of human activity
General Issues of Measurement Science
Logical and Mathematical Fundamentals of Measurement
Science
Emerging Fields of Measurement Science and Technology
Intelligent and Virtual Instrumentation
Education and Training
Potential authors are invited to submit an extended abstract
of up to four pages in PDF format by uploading the file to the
symposium web site, which contains detailed instructions. The
deadline for extended abstracts is February 15, 2008; notifica-
tion of acceptance will be by April 14, 2008; and the camera-
ready paper submission date is June 1, 2008.
For more information, please visit: imeko2008.scientific-
symposium.com
16th IMEKO Symposium on Electrical and
Electronic Measurements
The 16th International Measurement Confederation (IMEKO)
TC-4 Symposium Exploring New Frontiers of Instrumentation
and Methods for Electrical and Electronic Measurements will
be held in Florence, Italy on September 22-24, 2008.
The Symposium is a venue for the presentation of new ideas,
methods, principles, instruments, standards and industrial
applications on electric and magnetic quantities as well as their
diffusion across the scientific community. Participants will have
an excellent opportunity to exchange scientific and technical
information with specialists around the world and to enhance
Continued on page 14
METROLOGY NEWS
their international co-operation. The topics include:
Direct Current and Low Frequency Measurements
Radio Frequency, Microwave and Millimeter Wave
Measurements
Optical Wavelength Metrology
Calibration, Metrology and Standards
Traceability and International Compatibility of Measurements
Advanced Instrumentation Based on Micro and Nano Tech-
nologies
Digital and Mixed Signal Processing
Waveform Analysis and Measurement
Software Measurements
Biomedical Measurements
Dielectric Measurements
Power and Energy Measurements
Power Quality Assessment
Time and Frequency Measurements
Automated Test and Measurement Systems
Sensors and Transducers
Measurement for System Identification and Control
Virtual Measurement Systems
E-learning and Education in Measurements and Instrumen-
tation
Original contributions are warmly welcome in the form of
three to four page length extended abstracts.
The deadline for the submission of abstracts is February
29, 2008.
For more information, please visit the conference web site:
www.imeko2008.org
2008 Measurement Science Conference
The 2008 Measurement Science Conference (MSC) will be held
at the Disneyland Hotel in Anaheim, California from March 10
to 14, 2008. The following is from the
2008 Presidents Message by Miguel
Cerezo: As people across the nation,
and indeed the world, continue to
benefit from advances in measurement
technology, MSC progressively seeks to
improve on its programs and opportunities. This years theme,
Measure for Success builds on this concept and provides the
framework for goal achievement within the context of a contin-
uously improving and dynamic learning and business environ-
ment. Whether you come to learn the latest in measurement
techniques from NIST and industry professionals, seek to
immerse yourself in the most up to date information regarding
laboratory management or accreditation, wish to pursue
becoming an ASQ Certified Calibration Technician or simply
conduct a business meeting with your peers, suppliers or
clients, the Measurement Science Conference promises to offer
you a memorable and satisfying experience.
The annual Measurement Science Conference was founded in
1970 to promote education and professionalism in measure-
ment science and related disciplines. MSC has grown and
matured to meet the needs of dynamic measurement technolo-
gies as well as to address pertinent national and global measure-
ment issues. Based in California, the MSC has attracted experts
from around the world as speakers, exhibitors and attendees.
For more information on the Technical Program, Tutorial,
and Workshop, please visit: www.msc-conf.com
ANAB Expands to Offer Lab Accreditation
The ANSI-ASQ National Accreditation Board (ANAB)
announced on September 25, 2007, that they have acquired
Assured Calibration and Laboratory Accreditation Select Serv-
ices, LLC (ACLASS). This acquisition
expands ANABs range of conformity
assessment services to include accred-
itation of testing and calibration labo-
ratories. ANAB is the U.S. accreditation
body for management systems and is located in Milwaukee, WI.
ACLASS is located in Arlington, VA.
With the acquisition of ACLASS, ANAB adds to its existing
programs accreditation of laboratories to ISO/IEC 17025,
inspection bodies to ISO/IEC 17020, and reference material
producers using ISO Guide 34. ANAB accredits certification
bodies (CBs) for ISO 9001 quality management systems (QMS)
and ISO 14001 environmental management systems (EMS), as
well as numerous industry-specific requirements.
ACLASS is internationally recognized by ILAC, APLAC, and
IAAC through the signing of multilateral recognition arrange-
ments. These arrangements facilitate the acceptance of test and
calibration data between ACLASS-accredited laboratories and
the international community.
ANAB cooperates with other accreditation bodies around the
world to provide value to its accredited CBs and their clients,
ensuring that accredited certificates are recognized nationally
and internationally. The global conformity assessment system
ensures confidence and reduces risk for customers engaging in
trade worldwide.
Under terms of the acquisition, the two ACLASS principals
and current staff and experts will be employed by ANAB and
current ACLASS assessors are expected to provide ongoing
service to existing and new clients. There will be no change to
the ILAC recognition process.
For more information about ANAB, visit: www.anab.org. For
more information about ACLASS accreditation, visit:
www.aclasscorp.com.
SPECIAL REPORTS
1. Introduction
The purpose of this paper is to highlight some challenges that
metrologists and developers will be experiencing as new tech-
nologies are exploited and new systems are developed with mil-
itary and Global War on Terrorism (GWOT) related
applications. The intent is not to present an all-inclusive list of
these challenges, but rather to highlight a few examples that the
author considers to be particularly challenging or representative
of important applications within the DoD for which metrology
support is either deficient or nonexistent.
2. Aging Workforce and Metrology Training
It is no secret that a significant portion of the experienced
metrology and calibration workforce in the U.S. has either
recently retired, or will be eligible for retirement in the next few
years. This is especially true for the senior technicians, senior
metrologists, and managers in the metrology community, since
they earned their titles after accumulating years of on-the-bench
technical or managerial experience in the laboratory. The senior
staff represents years of corporate knowledge that is very diffi-
cult to replace. The 2005 NCSLI Benchmarking Survey [1] indi-
Electromagnetic Metrology Challenges
in the U.S. Department of Defense
Larry W. Tarr
Abstract: The Global War on Terrorism and the events that continue to unfold around the world are creating inter-
esting and innovative new developments in practically every area of technology. Many of the new developments are in
systems and components operating in the RF microwave and millimeter-wave portions of the electromagnetic spectrum.
The need to provide traceable metrology and calibration support for modern communications, radar, and smart
weapons systems operating at frequencies from a few kilohertz to 100 GHz and beyond presents challenges that con-
tinue to arise as new systems and technologies are developed. This paper will discuss some examples of the support
challenges facing the U.S. Department of Defense metrology community, and attempt to identify areas in which defi-
ciencies currently exist or are expected to develop.
Larry W. Tarr
U.S. Army Primary Standards Laboratory
AMSAM-TMD-S, Building 5435
Redstone Arsenal, AL 35898-5000 USA
Email: Larry.Tarr@us.army.mil
SPECIAL REPORTS
cated that at least 60 percent of the participating laboratories
used in-house resources to train their staff. As the highly-trained
and experienced metrologists and technicians retire in increas-
ing numbers, this valuable resource for on-the-job, in-house
training and expertise will be increasingly hard to find.
Traditionally, the largest source of trained calibration techni-
cians for many laboratories, including the Department of
Defense (DoD) service laboratories, has been retired Army,
Navy and Air Force personnel who were trained in the military
and served as technicians; electronics repairmen; radar opera-
tors and repairmen; test, measurement and diagnostic equip-
ment (TMDE) specialists; or a host of other related occupational
specialties. Since most of the military calibration training has
either been terminated or severely reduced, the pool of retired
military calibrators available for rehire into the civilian work-
force has diminished significantly in recent years. Experienced
calibrators with a broad knowledge of instrumentation, meas-
urement standards and good measurement practices are becom-
ing increasingly difficult to find. There are a few colleges that
offer specialized training in metrology and calibration skills. A
recent listing on the NCSLI education website included at least
13 institutions that offered degree or certificate programs in
metrology or a measurement technology area. Also, a growing
number of companies offer training in uncertainty analysis, sta-
tistical process control, accreditation topics, or specialty classes
in specific measurement parameters. However, training in RF,
microwave and millimeter-wave (MMW) metrology and calibra-
tion techniques is particularly lacking. Personnel with broad
knowledge and experience in these areas are especially difficult
to find. This indeed presents a major challenge to metrology lab-
oratory managers trying to build or sustain a technical staff with
expertise in RF, microwave and MMW measurements.
3. Funding Constraints
Since the tragic events of 2001, the Global War on Terrorism
has generated many programs designed to develop new tech-
nologies, systems and methodologies for dealing with real or
perceived threats to the U.S. The GWOT has created entirely
new government agencies and offices, with associated funding
requirements that in most cases have been supported by Con-
gress. In addition, since 2003 Operation Enduring Freedom
(OEF) and Operation Iraqi Freedom (OIF) have required huge
amounts of military funding that in some cases were obtained at
the expense of military organizations whose mission is sustain-
ment of metrology and calibration operations and TMDE. This
has indeed presented significant challenges to military calibra-
tion laboratories trying to provide the high-priority support for
soldiers, sailors and airmen in harms way, and at the same time
continue to meet the demands of their regular customers timely,
quality services. In some cases, military laboratories have been
required to provide only mission-essential services for extended
periods of time. In such instances, all other types of expendi-
tures, such as travel, training, overtime, contracts, procure-
ments, and developmental projects, were either cancelled,
delayed, or severely restricted. This environment presents signif-
icant challenges and impediments to the smooth operation of
military calibration laboratories.
4. Traceability Challenges
By regulation, every measurement performed by the DoD is
traceable through the Services calibration hierarchies to the
National Institute of Standards and Technology (NIST), or to
fundamental physical constants. Traditionally, that traceability
has been maintained by periodically submitting artifact stan-
dards to NIST for calibration, or by various Measurement
Assurance Programs (MAPs), standard reference materials, or
special measurements. As new technologies were developed, or
the state of the art improved, NIST responded accordingly by
developing new measurement systems, standards or techniques,
or by improving existing measurement systems, standards and
techniques. Generally speaking, NIST had the personnel
resources for responding to new measurement requirements,
although the development and characterization of a completely
new measurement standard typically takes from 5 to 7 years.
Also, if new measurement requirements were broad-based and
contributed significantly to the U.S. economy, NIST provided or
obtained funding to do the work. Conversely, if the military had
measurement requirements unique to the DoD, the military pro-
vided the funding. In either case, NIST generally had the
resources necessary to provide measurement traceability when
needed.
In recent years, NIST has lost a significant portion of its elec-
tromagnetic metrology technical staff. These include the much-
decorated, widely-published senior metrologists who played a
significant role in developing and establishing the U.S. metrol-
ogy system in use today. These also include the master machin-
ists, researchers, system developers, engineers, physicists,
technicians, and other personnel directly involved with provid-
ing the NIST calibration services so essential to disseminating
traceable measurements to NIST customers. Unfortunately, due
to funding restrictions, NIST has not been able to sustain or
replace many of their key staff positions. This has resulted in a
serious backlog in NIST calibration services. In some parame-
ters it has forced NIST managers to take a critical look at NIST
processes and to consider alternate methods for providing trace-
able measurements to the metrology community. Some of the
schemes under consideration include off-loading NIST measure-
ment services in areas with minimal workload. Magnetic field
intensity and 30 MHz attenuation are example parameters that
are now supported by the Navy and Army primary standards
laboratories, respectively. Other examples can be found in the
NIST Special Publication 250 series describing NIST measure-
ment services. Providing sufficient funding for NIST to ade-
quately support the metrology community remains a major
challenge.
5. Technological Challenges:
GWOT and Homeland Security Examples
5.1. MMW and Terahertz Imaging
In this application, new scanning systems are being developed
that can at least partially penetrate clothing and still provide suf-
ficient spatial resolution to reveal concealed weapons, including
those containing nonmetallic materials. (see [2], page 46) These
systems are being considered for deployment to airports and
SPECIAL REPORTS
other points of entry. They are also being tested for possible
application in detecting mines and explosives in tactical scenar-
ios. At present, there are few if any standards for quantifying the
spatial resolution, contrast, clothing penetration, and other sub-
jective aspects of image quality that affect the usefulness of
these scanning systems. Also, there are no quantitative measures
by which to intercompare these systems. The challenge will be
to develop standard objects, obscurants, scenarios, and test
systems to compare measurement results to simulations, and to
compare measurement results from different systems with one
another. The challenge will also be to characterize the perform-
ance of concealed weapon detection systems over a range of
conditions. At present, NIST provides few, if any, measurement
services at the MMW and sub-MMW frequencies at which these
scanners operate. Applicable measurement parameters include
power and noise-equivalent temperature difference.
5.2. RFID Technologies
The U.S. Government has mandated the adoption of radio fre-
quency identification (RFID) systems for identifying and track-
ing government assets. (see [2], page 47) However, lack of
metrology support is delaying the implementation of that direc-
tive. The impact is huge, impacting security and inventory
control at government sites, security at U.S. ports and ports of
entry, and operations of many high-volume retailers. For
example, the U.S. expects to issue at least 14 million passports
with RFID E-passports per year at a cost of at least $1.4
billion. The U.S. Government also plans to issue at least 4
million badges with RFIDs to government workers and contrac-
tors. Before these are deployed, standardized test parameters
need to be established to test readers for interoperability and
security, such as shielding against eavesdropping and the sus-
ceptibility of RFID tags to remote activation. The challenge to
metrologists will be to provide suitable calibration of load mod-
ulation (i.e., the signal returned from the RFID chip) and the
RFID readers electromagnetic fields to ensure consistent, reli-
able and secure operations of these devices.
5.3. Ultrafast Electronic Communications
Many of the new and emerging systems utilized by the DoD and
in the GWOT require synchronized ultrafast electronic commu-
nications. (see [2], page 195) Many of these systems utilize
technologies that require ultra-low noise measurements. These
systems include surveillance systems, ultra-high-speed comput-
ing, advanced communications, novel imaging systems, and new
defense ranging and positioning systems. The challenge to
metrologists is that the required ultra-low noise performance
can only be achieved and verified by substantially improving the
state-of-the-art in noise measurement, up to 10,000 times more
sensitive than is currently available anywhere in the world. The
frequencies of interest span 5 orders of magnitude, from the
upper microwave region (10
10
Hz) to the optical region (10
15
Hz). Ultralow-noise measurements are required to determine if
performance specifications are being met to ensure proper oper-
ation of these systems. Researchers are developing new ways of
measuring ultralow noise in systems based on the technology of
femtosecond laser frequency combs.
1
These combs can gener-
ate extremely precise signals from the microwave to the optical
range with very good control. In fact, the potential for these
combs to make microwave noise measurements nearly 10,000
times more sensitive than the current state-of-the-art in elec-
tronic measurements has been demonstrated recently. However,
much more research and development is needed to transform
this promising technology to reality.
6. Technological Challenges: Military Examples
6.1 Signal Proliferation
According to Richard Russell, the Chief of the Office of Science
and Technology Policy in the Executive Office of the President,
At some point very soon, were going to have a tremendous
spectrum crunch. [3] The problem is caused in part by the pro-
liferation of electromagnetic signals used by the military, and by
the fact that the military does not have a single authoritative
source to track the use of radio frequencies. The problem is par-
ticularly acute in Operation Enduring Freedom and Operation
Iraqi Freedom in the Middle East. Frequency assignments can
be made dynamically as new technology comes online. As a
result, conflicts can develop between wideband wireless com-
munications, radios, radars, and hostile devices such as Impro-
vised Explosive Devices (IEDs). The DoD is spending millions
of dollars to develop IED jammers of various types, which
further complicates the issue. The challenge for metrologists
and especially systems developers is to be cognizant of potential
interference and the implications to personnel, operations and
systems, and to develop new frequency allocation standards for
managing electromagnetic signals in military scenarios and
crowded urban environments.
6.2 Emerging Sub-MMW Systems
The development of satellite communications, radar, and
defense remote sensing systems in the frequency range from 110
to at least 500 GHz is currently inhibited by the lack of antenna
parameter metrology support. (see [2], page 48) Antenna
parameters (e.g., gain, pattern, and polarization) are critical to
optimizing, diagnosing, maintaining, and verifying the perform-
ance of these systems. It is estimated that reducing overall gain
uncertainty from approximately 2.8 dB to at least 2.2 dB would
improve weather forecasting and tracking, and result in an
implied benefit gain of $700 million per year to the U.S.
economy. Long term goals of further reducing radiometer uncer-
tainty to the 1 dB level would result in an estimated benefit
closer to $1 billion in 2003 dollars. The challenge to metrolo-
gists will be to develop and bring on-line a planar near-field
scanning system at NIST to characterize emerging sub-MMW
antennas with uncertainties approaching 1 dB.
1
Editors Note: Detailed information on the calibration of optical fre-
quency combs is included in the paper published in this issue of
Measure: Jack Stone, Liang Lu, and Patrick Egan, NIST, Calibrating
Laser Vacuum Wavelength with a GPS-based Optical Frequency
Comb, pp. 2838.
SPECIAL REPORTS
6.3 Ultrawideband Microwave Power Standards
With the adoption of standard 2.4 mm and smaller coaxial con-
nector geometries, several manufacturers have been marketing
or developing wideband power sensors with bandwidths of
50 GHz and higher. Providing accurate, traceable calibrations
of these devices is difficult, often involving several thermistor
mount working standards, characterized precision adapters, and
possibly combinations of waveguide and coaxial connections.
There have been attempts at developing precision power stan-
dards suitable for use as NIST-calibrated transfer standards. For
example, a NIST project funded by the military in the late 1990s
resulted in a limited number of power standards utilizing thin-
film technology and useable at frequencies from 10 MHz to
50 GHz, with 2.4 mm coaxial connectors. However, the produc-
tion yield of these sensors was extremely low, and no more
funding has been identified to pursue a follow-on project to
build more. The metrology challenge is to develop an improved
ultrawideband coaxial power standard, possibly calibrated
directly in a NIST-characterized microcalorimeter, at frequen-
cies from a few megahertz to at least 50 GHz.
6.4 Advanced Microwave and MMW Seekers
and Other Devices
Work continues in military and corporate R&D laboratories to
develop advanced seekers and other devices exploiting various
portions of the microwave and MMW spectrum. Applications
include smart munitions, target recognition and designation
systems, secure communications, ultra-high speed data trans-
mission, video distribution, MMW radios, portable radar
systems, and a multitude of other sensing, detection, and
imaging applications. The steady migration to higher frequen-
cies, coupled with the wide variety of application areas, presents
many interesting measurement and standards-related chal-
lenges. Some examples include requirements for full-band trace-
ability and measurement capabilities for power, thermal noise,
phase noise, and antenna characterization at frequencies of
110 GHz and higher New or improved standards and measure-
ment techniques may also be required for on-chip testing of
MMICs, measurements of scattering parameters and materials
properties of devices, and measurements of high continuous and
pulsed power at microwave and MMW frequencies. The chal-
lenge to metrologists is to keep up with the development and
potential deployment of these devices, and to ensure that suit-
able measurement standards and metrology support are in place
when required.
6.5 Synthetic Instrumentation
An increasing trend in RF and microwave metrology is the use
of instrumentation that define their functionality and capabili-
ties through software. Although such virtual instruments have
been around at least 20 years, their capabilities are advancing
rapidly and the DoD, as the largest single purchaser of test
equipment in the world, is beginning to exploit this technology.
Maintaining the militarys huge inventory of test equipment has
proved to be a significant and expensive challenge. One solution
was articulated by the DoD Office of Technology Transition [4]:
Recent commercial technology allows for the development of
synthetic instruments that can be configured in real time to
perform various test functions. A single synthetic instrument
can replace numerous single-function instruments, thereby
reducing the logistics footprint and solving obsolescence prob-
lems. The DoD recently formed the Synthetic Instrument
Working Group to address the issue and develop standards for
interoperability of synthetic instruments. With significant DoD
involvement, synthetic instrumentation will likely become the
wave of the future. Synthetic instrumentation already presents
challenges to the calibration community. Understanding the full
range of capabilities of these systems can be daunting. There can
be a steep learning curve associated with developing the expert-
ise to configure, operate and maintain these systems. In addi-
tion, questions arise about what needs to be calibrated and/or
verified in synthetic instruments, and what happens to the cal-
ibration when the instrument is reconfigured. All of these issues
represent significant challenges to metrologists as instrumenta-
tion continues to evolve from traditional rack-and-stack to more
versatile but complicated synthetic instruments.
6.6 MMW Scanning of the Surface Finish of Aircraft Skins
Work continues on a novel quantitative technique to measure
the macroscopic RMS surface roughness of surfaces made from
conductive and composite materials. By measuring the complex
amplitude variations in the speckle pattern of MMW signals
scattered from rough surfaces, the nature of the roughness can
be quantified directly. [5] This technique has been demonstrated
at 60 GHz and has potential broad applications in measuring
wear and corrosion of various surfaces, such as the interior sur-
faces of pipelines, vessels and aircraft skins. The challenge to
metrologists is to quantify the correlation between observed
reflected MMW speckle patterns and the actual RMS surface
roughness.
The technique also has applications in sensitive surface vibra-
tion measurements of optically rough materials. Since the
signal-to-noise ratio of such vibration measurements increases
as the square root of the amount of reflected light detected, and
since materials whose surfaces appear optically rough and
diffuse in visible light may appear much more reflective at
MMW frequencies, vibration measurements of optically diffuse
surfaces can be greatly improved by utilizing MMW systems
instead of optical interferometers. The challenge to metrologists
is to identify or develop improved high-speed MMW sensors
with sensitivities comparable to existing photon sensors in wide-
spread use in optical interferometry and vibration analysis.
6.7 Complex On-Chip Testing at Microwave
and MMW Frequencies
As military systems become more and more complex, RF Inte-
grated Circuits (RFICs) and Monolithic Microwave Integrated
Circuits (MMICs) are proliferating in applications ranging from
radios and wideband communications, to practically anything
employing electronic circuitry. RFICs are typically based on
CMOS or similar technologies, and their operation until
recently has been limited to a few gigahertz. MMIC devices are
SPECIAL REPORTS
typically based on gallium arsenide or similar technologies,
which have many proven applications at much higher frequen-
cies, including the MMW bands. Testing of these devices is
becoming more and more complex, requiring sophisticated mul-
tiport on-chip probing stations, complex network analyzers and
associated instrumentation, precision miniaturized measure-
ment standards and components, and a highly trained staff to
perform these measurements. Complicating the testing is the
continuing push toward RFICs and MMICs operating at higher
frequencies, even into the MMW bands. The challenge to
metrologists is to acquire the expertise to accurately character-
ize, operate and maintain these systems, the ability to develop
suitable measurement procedures and test plans, and the expert-
ise to analyze, interpret and report the results obtained.
7. Conclusions
New developments continue to unfold, almost on a daily basis,
in military uses of the RF, microwave and MMW spectrum in
the Global War on Terrorism. Staying abreast of these new and
emerging technologies and applications is a major challenge for
the metrology community. Developing the required metrology
and calibration support solutions can be a daunting task. Even
more fundamental are the issues related to training a new gen-
eration of metrologists and calibrators capable of meeting these
challenges. In this brief paper, we have presented a few exam-
ples of the challenges facing the military metrology community,
and we have attempted to identify areas in which deficiencies
currently exist or are expected to develop.
8. Acknowledgments
The authors would like to thank the reviewers of this paper for
their critique and helpful comments.
9. References
[1] Wade Keith, Monica Soltis, Nicholas Tyma, and Craig Gulka,
2005 NCSL International Benchmarking Survey, NCSLI
MEASURE, vol. 1, no. 1, pp. 1823, March 2006.
[2] An Assessment of the United States Measurement System:
Addressing Measurement Barriers to Accelerate Innovation,
NIST Special Publication 1048, Appendix B, Feb. 2007. (Avail-
able at http://usms.nist.gov/usms07/index.html)
[3] D. Perera and G. Grant, Managing Technology Mixed Signals,
Government Executive, pp. 6162, March 1, 2007. (Available at
www.govexec.com/features/0307-01/0307-01admt.htm)
[4] E. Starkloff, Creating a Synthetic Virtual Instrument for Avion-
ics Testing, Defense Tech Briefs, pp. 811, April 2007. (Available
at www.defensetechbriefs.com/content/view/1004/36/)
[5] J.B. Spicer, Microwave Metrology for Nondestructive Evalua-
tion, Materials Science and Engineering Dept., Johns Hopkins
Univ., Sept. 1998 (Available at www.jhu.edu/~opticnde/Doug/
microwav.htm)
JOE D. SIMMONS
MEMORIAL SCHOLARSHIP
The
Founded in the memory of Joe Simmons
to support the study of metrology
and metrology-related quality topics.
Completed applications are due March 1.
For application forms or more information contact your
advisor, student aid office, or the Scholarship itself at
Simmons_Scholar@comcast.net
www.simmons-scholarship.com
or write to: Simmons Scholarship
7413 Mill Run Drive, Derwood, MD 20855-1156
PROMOTING ACADEMIC EXCELLENCE IN METROLOGY
ASQ Measurement Quality Division NCSL International Measurement Science Conference
Outstanding students
are encouraged to apply for
the $3000 scholarship.
SPECIAL REPORTS
1. Introduction
In 1999, the International Committee for Weights and Measures
(CIPM) drew up an arrangement for the mutual recognition of
national measurement standards and of calibration and measure-
ment certificates issued by National Metrology Institutes (NMIs),
the CIPM Mutual Recognition Arrangement (MRA) (available at
www.bipm.org/en/cipm-mra/), with the objectives of:
Providing international recognition of national measurement
standards maintained by NMIs;
Providing confidence in, and knowledge of, the measurement
capabilities of participants for all users, including the regula-
tory and accreditation communities;
Providing the technical basis for acceptance, between coun-
tries, of measurements used to support the trade of goods and
services, as a result of the world wide acceptance of certifi-
cates issued in the framework of the CIPM MRA, and thus
Ultimately reducing non-tariff or technical barriers to trade.
Such a structure constituted an important step towards
improving the international metrology system and the overall
traceability of measurements to the International System of
Units (SI, available at www.bipm.org/en/si/). As such, the
CIPM MRA was welcomed by all sectors of the metrology com-
munity, by commercial and industrial companies, and by regu-
latory and accreditation bodies.
The CIPM MRA is an arrangement signed by metrology insti-
tutes. It can, however, have a positive impact only if quantita-
tive information supporting international recognition is
provided in an open and clear manner. Consequently, the CIPM
MRA created a database, maintained by the Bureau Interna-
tional des Poids et Mesures (BIPM) and known as the BIPM
key comparison database (BIPM KCDB). The KCDB openly
displays, on the internet, data on international comparisons of
A Users Guide to the Information in
the BIPM Key Comparison Database
C. Thomas and A.J. Wallard
Abstract: The launch of the Mutual Recognition Arrangement by the International Committee for Weights and Meas-
ures (CIPM MRA) created a process within which calibration and measurement certificates from National Metrology
Institutes (NMIs), which are signatories, could be recognized and accepted worldwide. This process has become of great
interest to regulators and accreditors. More recently, it has attracted the attention of international companies who wish
to take advantage of the mutual recognition offered by these certificates by realizing traceability to the International
System of Units (the SI) through local NMIs. This latter aspect of the use to which the BIPM key comparison database
(KCDB) can be put has recently been made more straightforward as the result of a new search engine installed by the
BIPM. This paper describes the current situation and shows how to access relevant information from the data base.
C. Thomas
A. J. Wallard
Bureau International des Poids et Mesures (BIPM)
Pavillon de Breteuil, F-92312 Svres Cedex
France
Email: cthomas @bipm.org
SPECIAL REPORTS
the national measurement standards maintained by participants
in the CIPM MRA, and on the Calibration and Measurements
Capabilities (CMCs) declared by these institutes.
The following section of this paper describes the content of
the BIPM KCDB and its impact on the worldwide metrology
system. The last section gives an overview of the search facili-
ties provided by the BIPM KCDB website, especially the text-
based semantic search engine, recently acquired and
implemented by the BIPM.
2. BIPM KCDB: Definition and Content
The BIPM key comparison database provides electronic support
to the CIPM MRA. The expression key comparison database
and the acronym KCDB are generic terms, which cover a
complete web application maintained by the BIPM. It is based
on two main databases and openly displayed on the internet
through the KCDB home page at www.bipm.org/kcdb. All the
information it contains is internationally recognized through the
procedures described in the text of the CIPM MRA, and kept up
to date by the BIPM staff.
Access is provided to three different types of information:
1. A list of participants in the CIPM MRA;
2. Information on, and reports of, key and supplementary
comparisons, including results interpreted in terms of
equivalence for key comparisons; and
3. Lists of Calibration and Measurement Capabilities (CMCs)
declared by the laboratories which participate in the CIPM
MRA, and which are internationally recognized by all other
participants.
2.1 Participants in the CIPM MRA
The participants in the CIPM MRA are metrology institutes that
are either:
National Metrology Institutes from Member States of the
Metre Convention or Associates of the General Conference
on Weights and Measures (CGPM), and whose Directors
have signed the CIPM MRA; or
Designated Institutes (DIs) nominated by the signatory NMI
as holding the national standards or facilities in the country,
when these are not available at the NMI itself.
The list of participants in the CIPM MRA is officially main-
tained by the Directors Office at the BIPM. It is made available
in the form of a .PDF file and of searchable html pages kept on
the BIPM website (see www.bipm.org/en/cipm-mra/participa-
tion/signatories.html). In March 2007, it included about 180
metrology laboratories from 45 Member States and 20 Associ-
ates, plus two international organizations, International Atomic
Energy Agency (IAEA) and Institute for Reference Materials
and Measurements (IRMM). A link to this list is made available
from the KCDB home page.
2.2 Key and Supplementary Comparisons
A first database is maintained at the BIPM to keep records of
international (key and supplementary) comparisons of national
measurements standards maintained by participants. The corre-
sponding information is made publicly available from the Key
and Supplementary Comparisons website
1
(see kcdb.bipm.org/
appendixB/KCDB_ApB_search.asp). These comparisons are
selected and managed by the Consultative Committees (CCs) of
the CIPM and by the Regional Metrology Organizations
(RMOs).
The philosophy behind the design of these comparisons is to
improve efficiency and reduce costs through a careful selection
which decreases the number of items to be compared, but which
gives confidence in a wide range of measurements.
2.2.1 Definitions
A key comparison is one of the set of comparisons selected by
a Consultative Committee (CC) of the CIPM to test the princi-
pal techniques and methods in a specific field of measurement.
The protocol is agreed to by the members of the CC.
Key comparisons are denoted CIPM key comparisons if
carried out by either one of the CCs or by the BIPM, and RMO
key comparisons if carried out by one of the RMOs within its
region. An RMO key comparison can be initiated only if a CIPM
(CC or BIPM) key comparison with the same protocol has
already been approved.
The BIPM key comparisons are a series of bilateral compar-
isons between an NMI and the BIPM. These are special cases of
CIPMkey comparisons, and are especially useful when a facility is
kept uniquely at the BIPM, for instance the International System
of Reference (SIR) for use in measurements of radionuclides (see
www.bipm.org/en/scientific/ionizing/radionuclides/sir/).
It is possible that participation in a key comparison reveals
some problem(s) in the measurement system involved in one or
several laboratories. The CIPM MRA has foreseen the case of
subsequent bilateral comparisons to key comparisons, in
order to provide the opportunity to repeat measurements.
A supplementary comparison is one of the set of compar-
isons conducted by the RMOs to meet specific needs not
covered by key comparisons, including comparisons to support
confidence in calibration and measurement certificates.
Each comparison is concluded by the edition of a Final
Report, which is reviewed and approved by the appropriate
body (CC or RMO), posted onto the KCDB website, and also
published as a Metrologia Technical Supplement (see
www.bipm.org/metrologia/TechSupp.jsp). For key compar-
isons, numbers and graphs are also displayed from the KCDB
website, as explained in next section.
2.2.2 Results of Key Comparisons
Key comparisons are organized to form families of comparisons
in a specific field of measurement, as shown in Fig. 1. Each
family is centred on a CIPM key comparison, to which RMO key
comparisons and bilateral comparisons are linked.
Measurements obtained by laboratories participating in a CIPM
key comparison are interpreted in terms of equivalence as foreseen
in the text of the Technical Supplement to the CIPM MRA (see
www.bipm.org//utils/en/pdf/mra_techsuppl2003.pdf). The re-
sulting numbers and graphs (often designated as results) are in-
serted in the Final Report, and also displayed in the KCDB website.
1
This part of the KCDB is often referred to as Appendix B of the
CIPM MRA.
SPECIAL REPORTS
They are composed of:
The key comparison reference value,
either deduced through a statistical
analysis of the results or by other
means accepted by the participants;
The degree of equivalence of each
national measurement standard,
expressed quantitatively by two terms:
(1) its deviation from the key compar-
ison reference value and (2) the uncer-
tainty of this deviation, at a 95 % level
of confidence; and
The graph of equivalence, which is a
graphical representation of the set of
degrees of equivalence (the zero-axis
represents the key comparison refer-
ence value).
The next step is the linkage between
two key comparisons of the same family,
and thus the expansion of the results of
the CIPM key comparison. This linkage
can be established only if there is a
common participation of one or more
members in the CIPM key comparison
and the equivalent RMO key compari-
son. The linkage does not modify the
value and the uncertainty of the CIPM
key comparison reference value, which
remains unique and unaltered for the
whole family. It simply extends the set of
degrees of equivalence and the graph of
equivalence in order to give evidence of
the comparability between institutes that
have only participated in one of the com-
parison exercises. The expanded uncer-
tainty included in their degrees of
equivalence is, however, generally higher
than if they had been compared directly.
An example is given in Fig. 2.
2.2.3 Contents of the KCDB
As of 5 April 2007, 720 comparisons
were registered in the KCDB, among
which there were:
567 key comparisons (78 from the
BIPM, 288 from the CCs, and 201
from RMOs), and
153 supplementary comparisons.
On the same date, results were inter-
preted in terms of equivalence for 254
key comparisons, leading to the publica-
tion in the KCDB of the corresponding
Final Reports and of about 800 graphs of
equivalence. 76 Final Reports of supple-
mentary comparisons were also available.
2.2.4 Impact
The interpretation of results in terms of
equivalence requires that complete and
documented uncertainty budgets are
established by all of the participants. The
output of the process is the elaboration
of sets of degrees of equivalence, forming
quantitative and objective information,
which has been reviewed and approved.
No attempt is given in the KCDB to
judge the performance of any one partic-
ipant relative to the others on a scale
from worst to best, but comparability
Figure 1. Organization of CIPM and RMO key comparisons.
NMI participating in CC key comparisons.
NMI participating in CIPM and RMO
key comparisons.
NMI participating in RMO key
comparisons only.
NMI participating in BIPM key compar-
isons (series of bilateral comparisons with
a unique facility maintained at the BIPM).
NMI participating in bilateral key
comparisons.
International Organization participating in
bilateral key comparisons.
Figure 2. An example of the extension of a graph of equivalence for luminous responsivity
(PR-K3.b family). This graph shows degrees of equivalence expressed in percent (%),
obtained as the offsets of individual measurements to the key comparison reference value.
The uncertainty bars correspond to a coverage factor k = 2.
CCPR-K3.b participants
One value amended after the CC
key comparison
APMP.PR-K3.b participants
Participants in three subsequent
bilateral comparisons
RMO key
comparisons
BIPM
CIPM
key comparisons
RMO key
comparisons
RMO key
comparisons
RMO key
comparisons
L
N
E
-
I
N
M
I
F
A
N
M
I
A
H
U
T
M
S
L
K
R
I
S
S
N
I
M
N
I
S
T
N
P
L
N
R
C
M
E
T
A
S
O
M
H
P
T
B
S
M
U
V
N
I
O
F
I
B
I
P
M
M
S
L
C
M
S
/
I
T
R
I
K
R
I
S
S
N
P
L
I
S
P
R
I
N
G
S
i
n
g
a
p
o
r
e
K
I
R
I
S
S
S
P
R
I
N
G
S
i
n
g
a
p
o
r
e
U
M
E
3
2
1
0
-1
-2
-3
D
e
g
r
e
e
o
f
E
q
u
i
v
a
l
e
n
c
e
,
%
SPECIAL REPORTS
between pairs of National Metrology
Institutes (and Designated Institutes) is
established on a solid footing, thus
enhancing confidence at the highest level
of metrological measurements. This com-
parability also constitutes a key item of
information for the international
approval of Calibration and Measure-
ment Capabilities declared by NMIs.
2.3 Calibration and Measurement
Capabilities
A second database maintained at the
BIPM keeps the list of Calibration and
Measurement Capabilities (CMCs)
declared and accepted by the institutes
participating in the CIPM MRA. The cor-
responding information is made publicly
available fromthe Calibration and Meas-
urement Capabilities CMCs website
2
(see http://kcdb.bipm.org/appendixC/
default.asp).
In the framework of the CIPM MRA, a
CMC is defined as the ensemble of infor-
mation given in a calibration or measure-
ment certificate issued by a metrology
institute. It corresponds to a service nor-
mally offered to all clients. A CMC is
described by the measured quantity and
its range, and is characterized by an
uncertainty generally given at a 95 %
level of confidence, together with the
method or instrument used, the values of
the influence parameters, if any, and any
other relevant information.
The CMC data registered in the KCDB
are prepared by the institute itself, then
peer-reviewed through intra- and inter-
regional examination. They should be
supported by information from some or
all of the following sources [Document
JCRB-14/06(2a), see www.bipm.org/
utils/common/documents/jcrb/
AppC_criteria.pdf]:
Results of key and supplementary
comparisons;
Documented results of past CC, RMOor
other comparisons (including bilateral);
Knowledge of technical activities by
other NMIs, including publications;
On-site peer-assessment reports;
Active participation in RMO projects;
and
Any other available knowledge or
experience.
At the end of the review, they are
approved by the Joint Committee of the
RMOs and the BIPM (JCRB) for publica-
tion in the KCDB. In addition, since July
2005, all CMCs displayed by the KCDB
website have been covered by an appro-
priate Quality System, approved by the
RMO of the declaring institute. The con-
sequence of this process is the interna-
tional recognition by all participants in
the CIPM MRA of the measurement and
calibration services listed by each insti-
tute in the KCDB.
2.3.1 Contents of the CMC Database
As of 5 April 2007, the database con-
tained some 19 300 CMCs, among which
about 3 700 cover the field of Chemistry,
3 400 the field of Ionizing Radiation, and
6 000 the field of Electricity and Magnet-
ism. This number is expected to increase
since some metrology areas are not yet
fully covered, especially Time and Ther-
mometry. Figure 3 gives an overview of
the number of CMCs recorded in the
KCDB over the last six years. Detailed
statistics on the number of CMCs
approved per country and per metrology
area are available from the KCDB statis-
tics web page (see www.bipm.org/utils/
common/pdf/KCDB_CMCs.pdf).
The effective number of uncertainty
values published is estimated to be over
35 000, as a consequence of the imple-
mentation of uncertainty tables, which
may be used to describe the range of
uncertainty values that characterizes one
given CMC. The column and row head-
ings of the table contain the values taken
by two physical quantities (or influence
parameters) involved in the CMC. This
facility, which makes it possible to deliver
much more precise information, is exten-
sively used in the field of Electricity.
2.3.2 Impact
The impact of the formal approval of
information about CMCs can be summa-
rized as follows:
The NMI certificates supported by
CMCs published in the KCDB are
accepted worldwide by all partici-
pants at whatever expanded uncer-
tainty is stated in the KCDB.
The Calibration Certificate Statement
of equivalence and the CIPM MRA
Logo can be included in the certificate
(see Fig. 4).
Commercial and industrial companies
can use the services of any signatory if
they need a calibration.
3
This certificate
will be accepted by all other signatories.
2
This part of the KCDB is often referred to as Appendix C of the CIPM MRA.
3 An example on how the CIPM MRA assists international trade is given in the KCDB Newslet-
ter (No. 6, December, 2006) from the experience of Dr. Gun Woong Bahng, Korea Research
Institute of Standards and Science (KRISS), Korea: The CIPM MRA saved DSME up to $ 10
million. (see kcdb.bipm.org/NL/06/ DSME_case_study.pdf)
Figure 3. Evolution of the number of CMCs registered in the KCDB over the last five years.
Note: More than 800 CMCs were deleted from the KCDB in the period July to September
2005, because they were not covered by an approved Quality System (QS). About 60% of
these were reinstated in the KCDB in April 2007.
21000
20000
19000
18000
17000
16000
15000
14000
13000
12000
11000
10000
Mar Oct Mar Oct Apr Sep Apr Sep Apr Nov Apr Sep
2002 2002 2003 2003 2004 2004 2005 2005 2006 2006 2007 2007
10874
13728
14394
15889
17499
15179
17376
17062
18081
18512
19454
19941
Introduction
of
Uncertainty
tables
Deletion of
CMCs (lack
of QS)
SPECIAL REPORTS
The KCDB CMCs website is a tool that is useful for regulatory
and accreditation bodies when they need to have information on
uncertainty and traceability.
3. Searching the BIPM KCDB
The KCDB home page gives access to both of the websites on
Key and Supplementary Comparisons, and on Calibration
and Measurement Capabilities CMCs, as shown in Fig. 5. It
also provides a number of useful links, including statistics, fre-
quently asked questions (FAQs), glossary, and the KCDB
Newsletters page.
From October 1999 to March 2007, information was made
available through directed search facilities allowing users to
choose among a list of items or by downloading .PDF files.
Some users, however, commented that searching information
from the database of CMCs is sometimes difficult: one has first
to select a metrology area, and then to choose from items pre-
sented under the format chosen for the Classification of Serv-
ices drawn up for this metrology area. These items may be
instruments, such as in dimensional metrology, or quantities,
such as in electricity. This can be confusing and leads the visitor
to simply download one or another global .PDF files from
among those proposed, without using the search engine that
would have delivered a well-targeted answer.
To overcome this difficulty, and also to increase the visibility
of the BIPM web system, a group composed of three BIPM staff
(Webmaster, Information Technology (IT) Manager, and KCDB
Coordinator) studied the advantages of implementing a search
facility that would be able to interpret a text-based inquiry.
Several such search engines, all commercially available, were
compared and the BIPM purchased new software in December
2005. Our new search engine was implemented on the KCDB
and publicly launched on 6 March 2007. It takes the form of
free-text boxes, into which the user types words, available from
the websites which contain the information on comparisons and
the CMCs. The previous directed search facilities are also main-
tained for sake of continuity.
The BIPM search engine is also implemented on the main
BIPM website. It offers a search across the websites of all insti-
tutes participating in the CIPM MRA and a number of databases
maintained at the BIPM, so promoting the BIPM website as the
worlds reference portal for metrology.
The BIPM search engine is a powerful tool with the advan-
tages of full-text searching, and dynamically generated tables of
contents based on each search results page, to allow an easy
means of refining the search query, as shown in Fig. 6.
The following sections show some characteristics of the new
BIPM search engine through a number of illustrative examples
which the reader is encouraged to experience.
3.1 Relevance of the Results
The search engine parameters are chosen so the answers are all
appropriate (minimization of the noise) and no appropriate
answers are missing (minimization of gaps in the information).
For instance, the query AC-DC
5
in the database of CMCs
returns about 1000 answers, all relevant to AC-DC voltage and
current transfer at all frequencies.
3.2 Refining Search Results
The links generated dynamically on the left of the screen can be
used to refine the search by selecting or deleting one or several
item(s) among the proposed lists. It is always possible to come
back to the previous screen by clicking again on the same link.
In the example AC-DC, it is possible to choose an RMO or an
NMI, and measurements at radio-frequencies for instance.
Note that this type of web navigation requires only a few
clicks, and no a-priori knowledge.
3.3 Finding Comparisons Corresponding to a Specified
Theme
It is now possible to search for comparisons involving a speci-
fied theme. For instance, natural gas returns key and supple-
mentary comparisons in Chemistry and Fluid Flow.
3.4 Obtaining Statistics
Any type of statistics based on numbers of CMCs or on compar-
isons corresponding to specified properties is now facilitated.
For instance, searching for key and refining by selecting the
statuses Report in progress and Measurements completed
Figure 4. The CIPM MRA Calibration Certificate Statement and the
CIPM MRA Logo.
4
Calibration Certificate Statement
This certificate is consistent with the capabilities that
are included in Appendix C of the MRA drawn up by the
CIPM. Under the CIPM MRA, all participating institutes
recognize the validity of each others calibration and
measurement certificates for the quantities, ranges and
measurement uncertainties specified in Appendix C.
4 In April 2007 there were 55 laboratories authorized to use the CIPM
MRA Logo on the measurement and calibration certificates of their
services listed in the KCDB. Other laboratories that want to use the
logo can do so by application to the BIPM. (A full list of authorized
laboratories is available at www.bipm.org/en/cipm-mra/logo/autho-
rized_users.html)
5
Note that if the entry includes a slash (/), the search engine under-
stands it as a separator and considers only the first part of the entry.
It would thus dramatically truncate the answer. The use of a slash is
thus not recommended. The search engine is set up on the KCDB
website in such a way that the sign - and the hyphen are understood
as the word and.
SPECIAL REPORTS
returns those key comparisons that are
about to be finished. Another example
may be Electricity and Magnetism
United States which would return all
CMCs declared by that country and rele-
vant to that metrology area.
3.5 Approximation: Exact Wording
Search
An approximation of two letters on the
entry is allowed. It follows that the plural
is automatically taken into account: the
queries flowmeter and flowmeters are
thus equivalent. Exact wording search is
possible by using quotes; thus the queries
VNIIM and VNIIMS (two Russian
metrology institutes) are not equivalent.
3.6. Vocabulary: Use of Acronyms
A vocabulary has been implemented on
the search engine, which makes it possi-
ble to find information relevant to syn-
onyms of the entry. It is intended to be
especially useful when acronyms are
searched: for instance DVM for digital
voltmeters.
3.7 Searching a Certified Reference
Material or NMI Service
Though the KCDB is by no means a cat-
alogue of Certified Reference Materials
(CRMs), a number of CMCs, especially
in chemistry, note the use of CRMs for
dissemination of traceability. It is now
possible to call for a specific CRM, for
instance CASS-4, and access all meas-
urements that this particular CRM sup-
ports (in this case, a number of analytes
measured in sea water by Canada). It is
also possible to have direct access to a
specified NMI service known by its iden-
tifier, for instance PTB 94a available
from the German NMI.
4. Conclusions
The worldwide metrology system is well
supported by the Mutual Recognition
Arrangement drawn up by the CIPM and
its electronic support, the BIPM key
comparison database. The process put in
place helps ensure traceability to the SI
of measurements stated on calibration
certificates. It provides a means to vali-
date the traceability and the uncertainty
claims of commercial and industrial
companies and of laboratories world-
wide which have been accredited to
ISO/IEC 17025 or, in the case of refer-
ence materials, to ISO Guide 34.
Ensuring the maintenance and the
update of the KCDB, and providing
access to its content, are essential and
continuous tasks for the BIPM, which are
central to its fundamental mission. The
BIPM invites feedback from the KCDB
visitors, in order to further improve its
service to the widest possible user com-
munity.
5. Acknowledgements
Claudine Thomas extends warm thanks
to Janet Miles, the BIPM webmaster, and
to Laurent Le Me, the BIPM IT
Manager, for helpful collaboration in
implementing the BIPM search engine on
the KCDB website.
The BIPM is also grateful to the
National Research Council, NRC,
Canada for hosting the KCDB demon-
stration on their stand in the NCSLI
exhibition hall at the 2007 NCSLI Work-
shop and Symposium, St. Paul, MN.
Figure 5. The BIPM KCDB home page.
Figure 6. Results of the query copper in water entered in the free-text box implemented
on the KCDB CMCs website. The right part of the screen displays some of the relevant
CMCs. A number of contextual links are provided on the left and can be used to refine the
search by geographic location of the declaring NMI, and by chemical material.
TECHNICAL PAPERS
1. Introduction
Optical frequency combs [1, 2] provide a direct link between
typical frequency standards, which operate at frequencies in the
megahertz or gigahertz range, and the frequencies of visible
light, which are on the order of 10
14
Hz. They provide a method
for generating optical radiation whose frequency (or vacuum
wavelength) can be determined with exceptional accuracy. If the
difference frequency (beat frequency) is measured between a
comb component and a laser, then the frequency and vacuum
Calibrating Laser Vacuum
Wavelength with a GPS-based
Optical Frequency Comb
Jack Stone, Liang Lu and Patrick Egan
Abstract: The Global Positioning System (GPS) can deliver an exceptionally accurate frequency standard to any point
in the world. When the GPS signal is used to control an optical frequency comb, the comb + GPS system provides laser
light with well-known frequencies (or equivalently, vacuum wavelengths) over much of the optical spectrum between
0.53 m and 2 m. The comb vacuum wavelengths can serve as primary length standards for calibration of the wave-
length of metrology lasers, and the uncertainties of the comb wavelengths are sufficiently low to be suitable for almost
any imaginable task associated with length metrology. The GPS signal is traceable in the sense that its uncertainty
is continually assessed via measurements at NIST/Boulder, and results of the measurements (in effect, calibration
reports) are published on the web. Thus it can potentially deliver a traceable standard of unprecedented accuracy to
any laboratory, but how can the user be certain that the resulting laser calibrations have comparable accuracy? These
calibrations depend not only on the GPS signal but also on much additional equipment (including a disciplined oscil-
lator, optical frequency comb, and optics/electronics for beat frequency measurement), and any such system might
contain additional sources of error if it is poorly designed or operated by inexperienced personnel. However, it is shown
that internal consistency checks can be effectively used to verify the proper operation of the measurement system. In
many respects these internal consistency checks provide better confidence in the results than what is likely to be
achieved by more traditional methods of establishing traceability, such as sending an instrument or artifact to a national
metrology institute for calibration. A relative expanded uncertainty of less than 2 10
-12
can be verified for laser cal-
ibrations, and this is sufficient even for the most demanding applications of dimensional metrology.
Jack Stone
Liang Lu
Patrick Egan
National Institute of Standards and Technology
100 Bureau Dr, Mail Stop 8211
Gaithersburg, MD 20899 USA
Email: jack.stone@nist.gov
TECHNICAL PAPERS
wavelength of the laser can be deter-
mined. This provides a direct link
between the International System (SI)
definition of the second, as realized by
a frequency standard, and the wave-
lengths of stabilized lasers that form the
basis of most high-precision dimensional
metrology.
In the National Institute of Standards
and Technology (NIST) Gaithersburg
laboratory, a commercial optical fre-
quency comb [3, 4] is used to calibrate
lasers. The comb consists of a mode-
locked erbium fiber laser, an f 2f inter-
ferometer for determining offset
frequency, and frequency doubling optics
to reach visible wavelengths. (See
Fig. 1.) When the repetition frequency,
f
rep
, of the mode-locked laser is locked to
the frequency of a Global Positioning
System Disciplined Oscillator (GPSDO)
[4, 5], the output laser light then consists
of frequency components distributed in a
uniform manner, with frequencies, f
n
,
given by the equation:
(1)
where f
0
is the offset frequency and n is
a large integer (greater than 10
6
). The
operating principles of combs have been
described extensively in the literature
and will not be described here. (See ref-
erences [1, 2] for good reviews of work
on frequency combs. Reference [6]
explicitly discusses using a GPSDO in
conjunction with a comb for laser cali-
brations.)
When the offset frequency f
0
is stabi-
lized using the f 2f interferometer [1,
2], and if the integer n associated with a
particular frequency component f
n
can
be determined without ambiguity, then
the primary uncertainty in f
n
is the uncer-
tainty of f
rep
, which in turn is determined
by the uncertainty of the GPSDO refer-
ence. This uncertainty can be quite
small. For twenty minute averages, it is
estimated that the relative expanded
uncertainty of the GPSDO frequency is
1.6 10
-12
(with coverage factor k = 2).
Methods of finding n are discussed in the
literature. [1, 2, 7] For this paper it will
be assumed that n is known, although
there is a brief mention of this assump-
tion in Section 4.
A well-designed optical frequency
comb does not increase the uncertainty
beyond the uncertainty of the frequency
reference. Two combs that share the
same reference frequency have been
shown to have no appreciable offsets at a
level many orders of magnitude below
the levels of uncertainty discussed here.
It has been shown that combs agree with
each other even at the 10
-19
level! [8]
This is more than 5 orders of magnitude
smaller than the uncertainty of the GPS
frequency. Consequently, only a serious
blunder would prevent a user from
attaining the uncertainty characteristic of
the GPS signal.
Even though the GPS signal provides a
well-characterized standard with uncer-
tainty levels smaller than what is needed
for this application, there is always a
danger that some other part of the cali-
bration process might introduce errors
into a measurement. Consequently it is
necessary to adopt some strategy to
assure confidence in the results. There-
fore, the question that is addressed in
this paper is: How might a laboratory
verify that its laser calibrations are
indeed accurate at relative uncertainties
on the order of 10
-12
? This uncertainty is
small enough for almost any imaginable
need associated with dimensional
metrology, but is nevertheless very
modest relative to state-of-art comb
capabilities. The issues involved here are
somewhat different from the issues that
more typically dominate discussions of
assuring confidence in measurements
results. Below four methods are enumer-
ated that are commonly used to assure
the validity of dimensional measure-
ments, and a discussion is presented of
how these techniques relate to measure-
ments of vacuum wavelength (fre-
quency) with the comb + GPS system.
Method (1): It is always necessary to
develop an uncertainty budget for a
measurement process. Often the uncer-
tainty associated with the basic standard
is only a small part of the full uncertainty
of a measurement process, but for the
comb + GPS system, the basic standard
(the GPS signal) is the only significant
term in the uncertainty budget. As previ-
ously mentioned, state-of-art comb meas-
urements have demonstrated agreement
between combs at uncertainties ranging
from 4 to 7 orders of magnitude smaller
than the GPS signal, and in a well-
designed comb system there are simply
no noticeable sources of uncertainty
associated with the comb itself. The
current situation is somewhat analogous
to using a high-quality micrometer for
measurements of a 10 cm distance with a
desired uncertainty of 1 cm. In this case,
the development of a careful uncertainty
budget is not particularly helpful in ver-
ifying that the desired level of uncer-
tainty has been reached. The only
significant concern is that, as a conse-
f
n
= f
0
+ n f
rep ,
Figure 1. The optical frequency comb, shown in the foreground with the blue covers
removed, consists of three units: a fiber laser (center unit), an f 2f interferometer for stabi-
lization of the offset frequency (left), and a frequency-doubling unit to reach visible wave-
lengths (right). In the background is an iodine stabilized laser that was compared to the
comb.
TECHNICAL PAPERS
quence of some blunder associated with instrument design,
instrument failure, operator error, or non-optimal setup, the
measurement system is not functioning in accord with the
intended principles of operation. It is not appropriate to include
such possible blunders in an uncertainty budget, but it is nec-
essary to assure that these blunders do not occur.
Method (2): Calibration of equipment by an external labora-
tory (such as NIST) can be used to help justify the uncertainty
budget. In the case discussed here, the GPS signal, which serves
as the basic standard underlying the measurement, is calibrated
daily by NIST, in the sense that the signal is monitored and its
errors are assessed on a continuous basis. It is probably not nec-
essary to calibrate any additional equipment in the classic sense
of the word calibrate. For example, there is no need to peri-
odically compare the GPSDO to a cesium clock, as would be
required if we had a free-running rubidium or quartz oscillator
that drifted slowly over long periods of time. It is necessary,
however, to assure that a piece of equipment (such as the
GPSDO) is functioning in an appropriate manner, and we will
discuss how this might be done.
Method (3): Often the best way to check for hidden sources
of error is to participate in an interlaboratory comparison (a
round-robin) to demonstrate that all participants obtain consis-
tent measurement results for an artifact or instrument. For the
comb system, it will be argued that it is unnecessary to carry out
comparisons at uncertainty levels sufficiently low that they can
provide direct support for uncertainty claims that can be easily
justified based on other evidence. However, possible benefits
will be discussed in carrying out comparisons at uncertainty
levels worse than the demonstrated capability of the comb
+ GPS system.
Method (4): Studies of short-term repeatability or other inter-
nal consistency checks can play a pivotal role in verifying the per-
formance of many measurement systems, and this is particularly
true of the comb + GPS system. Sending ancillary equipment for
testing or participating in an intercomparison could be useful
activities, but we would argue that internal consistency checks
are the single-most important method of verifying that the comb
+ GPS system will perform at the claimed uncertainty. Imple-
menting internal consistency checks can provide much better
confidence in the results than would any single alternative
method of achieving confidence in results, because some of these
tests can be ongoing activities providing continuing evidence that
indeed everything is working properly. When appropriate
equipment is available, internal consistency tests can also
uncover most likely blunders of operation or of equipment
design. In fact, using this type of testing, a laboratory can be con-
fident that their system achieves their claimed uncertainty, even
though they employ uncalibrated commercial equipment which
in some respects is a black box due to limited documentation.
The remainder of this paper discusses why the authors are
confident of their results. It describes how a laboratory can
achieve a very high level of confidence in laser wavelength
measurements by quantifying uncertainty, by establishing a
close link to the SI definition of the second, and by checking for
operator errors or poor setup that might prevent the measure-
ment system from achieving its intrinsic high accuracy. Estab-
lishing confidence in measurements with well-quantified uncer-
tainty is closely related to the concept of traceability, but the two
concepts are not identical, and a discussion of traceability per
se will be avoided in this article.
2. Sources of Uncertainty
and How They Can Be Quantified
2.1 GPS Signal
The GPS signal averaged over suitably long periods of time has
a very low uncertainty. NIST monitors the GPS signal and
reports the results on the web at http://tf.nist.gov/service/
gpstrace.htm. As recorded in these archives, the fractional Allan
deviation for two-day frequency averages over the last 200 days
is under 3 10
-14
, and the Allan deviation decreases as the aver-
aging time increases. The long-term (multi-day) intrinsic GPS
errors are negligibly small for dimensional applications, where
uncertainties smaller than 10
-13
have no visible impact on
the work. On shorter time scales, the Allan deviation increases,
but even for 20-minute samples, the Allan deviation is only
1.3 10
-12
, which is very small compared to the uncertainty that
is typically assigned to an iodine stabilized laser and is sufficient
for almost any imaginable need of dimensional metrology.
The NIST archive is useful because it will show how well the
GPS signal is performing on a given day, recording possible
degradation of the signal due to solar storms or to bad satellites.
However, the archived information does not directly tell one the
uncertainty of the GPSDO frequency for short averaging times.
2.2 GPSDO
Traceability of GPSDO systems has been discussed widely in the
literature, and on the NIST web site. [9] (See also the traceabil-
ity discussion in reference [10].) In this paper the uncertainty
of the GPSDO frequency signal is discussed, which is one com-
ponent of traceability, but the other formal requirements asso-
ciated with traceability are not discussed.
It is important to distinguish between the intrinsic uncertainty
of the satellite signal and the uncertainty of the GPSDO. The
short-term uncertainty of a GPSDO depends not only on the
GPS signal but also on the quality of its local oscillator, on the
quality of the complex algorithms used to process the satellite
signals and steer the local oscillator, and on the antenna place-
ment. For the GPSDO installed at Gaithersburg, the observed
20-minute Allan deviation is a factor of two lower than what is
indicated by the NIST archives. At short averaging times, some
GPSDOs can provide better results than the archived results if
the units are of good quality, with a sufficiently stable local oscil-
lator, and well-designed algorithms are used for processing the
GPS signal and steering the oscillator.
It is also important to distinguish between the long-term
uncertainty and the short-term instability of the GPSDO. There
are several intrinsic time scales that might be used to distinguish
short-term and long-term performance. One natural time
scale would be one sidereal day, because the GPS satellites pass
overhead once per day, so that effects associated with individ-
ual satellites or with multipath reflections into the antenna will
repeat daily. This time scale also includes effects due to diurnal
TECHNICAL PAPERS
variations of the ionosphere and diurnal temperature variations.
Another important scale is set by time constants in algorithms
that steer the GPSDO local oscillator. In this article long-term
will be used to designate periods of time long compared to any
of these natural time scales. Usually the time constants in
GPSDO steering algorithms are no more than a few hours, so
short-term might naturally refer to times less than one day.
However, there are exceptional commercial systems where a
GPSDO might update steering of the local oscillator on a time
scale of weeks [11] and short-term or long-term must be
defined accordingly. This terminology is not in accord with
general usage, and the reader is warned that this non-standard
terminology will be used throughout the remainder of the
article. Given these definitions, all plausible GPSDO errors are
believed to be short-term and there are no long-term systematic
offsets.
Although it is true that some GPSDO units have much better
performance than others in terms of Allan deviation for a par-
ticular sampling interval, and many commercial units are simply
not well designed for use as a frequency standard, the authors
are aware of no instances where commercial units give system-
atic long-term frequency offsets. Therefore a study of short-term
repeatability will suffice to quantify all uncertainties associated
with the GPSDO, but a practical difficulty lies in knowing
exactly what time intervals correspond to short-term or long-
term. In establishing the duration of the test, one reasonable cri-
terion would be that the Allan deviation (or total deviation
[12]) should show a convincing decrease for the longest sam-
pling times. This might be a necessary condition but is not suf-
ficient. It is also desirable to find out from the GPSDO
manufacturer what is the longest time scale associated with the
unit. The testing period should surely exceed one day and
should significantly exceed the time scale given by the manufac-
turer. A minimal reasonable testing period might be at least
twice as long as any natural time scale of the system.
It is difficult to imagine a physical mechanism through which
long-term errors might arise. The most likely sources of a con-
stant, systematic offset would be either complete failure of the
GPS steering (discussed later) or a blunder of design, such as a
firmware bug that causes the device to not work as intended.
Metrologists usually assume that blunders simply will not occur
and thus efforts are concentrated on more subtle problems, but
for the GPSDO the only possible cause of systematic long-term
offsets would be a design blunder (or complete failure). Gross
firmware errors of a factor of two are easily discovered, but
more subtle errors might produce small offsets. It is assumed
that after updating the firmware or introducing a new model, a
competent manufacturer would take appropriate precautions to
assure that there are no systematic offsets in the new model.
Realistically it should be easy to assure that no software design
blunder has introduced an offset; this might be done either by
sending the new model to the national metrology institute for
testing or by comparing the new model to an older unit that was
previously shown to be free of such offsets. Thus, if modest and
reasonable care is exercised to assure the integrity of a new
model, it is difficult to see how a systematic offset would ever
occur, even at the 10
-14
level.
We must consider the possibility that individual units might
have errors even though the entire product line is not faulty. It
is not clear what the physical mechanism of such an error might
be, other than catastrophic failures where the GPSDO does not
function at all according to its intended principle of operation.
In most cases the GPSDO will provide diagnostics and will alert
a user if a catastrophic failure makes it impossible to steer the
local oscillator to the GPS signal. Most GPSDOs would display
some warning if satellite reception were lost (perhaps due to
intermittent RF interference, for example) but other failure
modes might elude the internal diagnostics and require an addi-
tional test to verify proper operation of the unit. Such cata-
strophic failures can be uncovered through tests that will be
described here.
Like any instrument, a GPSDO can be misused in a manner
that will yield bad results. For example, if the antenna is poorly
located so that it is subject to multipath reflections and has a
restricted view of the sky, short-term fluctuations will increase
(but there will be no systematic offset). For this application, a
more dangerous error would occur if the antenna cable were
completely disconnected or if RF interference made it impossi-
ble to receive the GPS signal, so that there is no GPS control of
the local oscillator, in which case the system could develop a
long-term offset. Similarly, an electronic malfunction in the
GPSDO or a malfunctioning rubidium cell might cause a com-
plete shutdown of the GPS steering, or in some cases it might
be possible for an operator error to effectively disable GPS
steering. This might result in a subtle offset that would not be
immediately obvious, However it is argued that, for GPSDO
systems similar to what is used here, even total failures of the
GPS steering can be uncovered by studying the short-term sta-
bility of the system.
In the absence of a blunder that would give rise to a long-term
systematic offset, short-term fluctuations in the GPSDO output
are the greatest concern. Short-term fluctuations might be quan-
tified by sending a unit to the national metrology institute or to
some other laboratory for testing, but quantifying short-term
fluctuations is not logically tied to the SI unit and, if a suitably
stable local frequency source is available, it is often preferable
to quantify short-term fluctuations with the equipment in situ.
In situ testing has the significant advantage that it can include
errors associated with a bad location of the antenna or other
issues associated with local installation of the apparatus. In
many cases short-term stability studies would also uncover the
possible installation blunders discussed previously in this
section. One useful approach is to compare the comb output to
an iodine stabilized laser and thus quantify the combined insta-
bilities of all system components, the GPSDO and the optical
frequency comb.
2.3 Stability Measurements
The short-term stability of the entire system can be evaluated by
comparing the comb output to an iodine stabilized laser. The
stability of an iodine stabilized laser over a period of one week
is much better than the absolute uncertainty of the vacuum
wavelength, and consequently the laser can be used to quantify
short-term fluctuations with an uncertainty that is small enough
TECHNICAL PAPERS
to be useful. To assure good results, the
laser should sit undisturbed in a temper-
ature-controlled laboratory and be given
many of hours of warm-up before begin-
ning a measurement. Reasonably good
stability has been observed under these
circumstances without the need to read-
just the laser operating parameters
(which might itself be a source of insta-
bility). Although the laser frequency
drift indeed limits the ability to charac-
terize the system, the amount of drift is
small enough to satisfy the present
needs.
Stability measurements were carried
out over a period of six days and the
measured Allan deviation is plotted in
Fig. 2. The measurement quantifies the
combined effect of short-term fluctua-
tions in the GPSDO, the comb, the
iodine stabilized laser, and all ancillary
equipment such as frequency counters.
During the period of time when these
data were collected, there was very little
sunspot activity and NIST did not record
any unusual errors in the GPS signal.
The current measurements thus provide
a baseline of what can be expected under
ideal conditions of the ionosphere, and
any possible future degradation of the
signal must be taken into account by
checking the NIST archives.
The shape of the curve is very similar
to the manufacturers published results
for this GPSDO, although the magnitude
of the observed Allan deviation is smaller
than shown by the manufacturer for
sample times longer than 10 s. (There are
many potential reasons why the meas-
ured Allan deviation might be lower than
published values, including the low solar
activity.) These results give some confi-
dence in this system; they strongly
suggest that the GPSDO is the limiting
element of the system at averaging times
in excess of 10 s, as it should be. For
samples of less than 10 s, there may be a
significant contribution to the Alan devi-
ation from instability in the iodine stabi-
lized laser. Also, there is a possibility that
drift of the laser is affecting results for
averaging times in excess of one day. The
measurements indicate the presence of
an overall drift of about 400 Hz over the
117 hour duration of the test (a frac-
tional change of 1.7 10
-13
per day).
This variation is likely associated with
instability of the test laser rather than
with the GPSDO or the comb. In reality,
the comb was compared to two iodine
stabilized lasers, and the second laser
appeared to drift in the opposite direc-
tion.
The shape of the curve shows the clear
signature of the manufacturers steering
algorithm for this GPSDO. This signa-
ture would not be present if the GPS
steering were somehow disabled by a cat-
astrophic failure.
There is structure in the Allan devia-
tion data for times as long as 10 000
seconds (2.8 hours), in agreement with
the manufacturers published measure-
ments. This suggests that time constants
associated with the GPSDO are at least
this long. Thus, it is argued that the six-
day testing period is more than suffi-
ciently long to sample any plausible
source of uncertainty, and this claim is
bolstered by the fact that the Allan devi-
ation falls off for the longer sample
times.
Of course, the possibility that the soft-
ware makes some steering corrections
once every two weeks or even once per
year can not be ruled out. Unusual
firmware routines may accidentally
defeat the intrinsic good stability of the
GPS signal for long averaging times. If
the GPSDO only applied corrections to
the rubidium oscillator once every two
weeks, and if the six-day test did not
overlap one of these corrections, then the
magnitude of fluctuations would be
underestimated. It is therefore advisable
to verify from the manufacturer of the
GPSDO that steering on such a long time
scale does not occur.
Figure 3 shows results for beat fre-
quency measurements between the comb
and laser, with 22 minute (1340 s) aver-
aging time, as measured over a period of
six days. The sample time of 1340 s is on
the plateau in Fig. 2. The frequency is
graphed as a function of the time of day
(Eastern Daylight Time). Although the
data are noisy, it suggests that there may
be a systematic correlation with time of
day, including a downward variation of
about 1 kHz between 19:00 and 3:00
and a 1 kHz upward variation between
3:00 and 6:00. Because of these system-
atic variations on a time scale of many
hours, the 22-minute Allan deviation will
slightly underestimate the true uncer-
tainty of a 22-minute measurement. The
long-term variations are reflected in the
fact that the Allan deviation does not
decrease greatly as the averaging time is
increased above 20 minutes (Fig. 2),
although substantial reductions in uncer-
tainty (more than a factor of 2 improve-
ment) are apparent when averaging over
periods of time in excess of six hours.
The sample standard deviation of the
data in Fig. 3 is 370 Hz (7.8 10
-13
), and
95 % of the points are within 730 Hz of
Figure 2. Allan deviation, comparing the comb to an iodine stabilized laser.
1 10 100 1000 10000 100000
Time (s)
F
r
a
c
t
i
o
n
a
l
A
l
l
a
n
D
e
v
i
a
t
i
o
n
10
-11
10
-12
10
-13
TECHNICAL PAPERS
the mean. This suggests that a reasonable
k = 2 estimate of the uncertainty for a
measurement averaged over 22 minutes
is 740 Hz (1.6 10
-12
) or less, where this
value might be slightly inflated due to
drift of the iodine stabilized laser. The
uncertainty assignment is only meaning-
ful because it has been shown that the six
days of observation is sufficient to
sample all significant sources of variabil-
ity in the system. Ignoring the drift of the
iodine stabilized laser, it is believed that
the six-day average of data such as in Fig.
3 is expected to be an unbiased estimate
of the true value with an uncertainty very
small compared to the claimed uncer-
tainty, and the 370 Hz standard deviation
of the data in Fig. 3 should be a true
measure of the uncertainty for 22-minute
sampling. Essentially the same sample
standard deviation is seen for 22-minute
averages whether it is measure for a
single day or for the six day period. If the
comb could be compared to a perfect sta-
bilized laser, and if sunspot activity were
constant, there would be every reason to
expect that a similar standard deviation
would be obtained if data were taken not
for six days but for six years.
In summary, it is believed that these
studies of stability over a six day period
are all that is needed to assess the uncer-
tainty of the GPSDO, other than the
additional requirement that one always
check the NIST web site to see if there
has been either some failure of the GPS
system or a solar storm. Although this
particular unit has never been independ-
ently tested, it is known that similar units
have been tested and consequently it is
believed that the entire product line is
not defective. Furthermore, the observed
stability plot of Fig. 2 shows clear evi-
dence that the GPS steering has not shut
off. In the absence of such a failure, there
is no plausible physical mechanism
through which an individual unit would
have a hidden long-term offset that is not
shared by the rest of the product line,
and consequently there is full confidence
in this unit, even though it has never
been independently tested.
The stability study above has elements
in common with the use of control stan-
dards to estimate uncertainties in dimen-
sional metrology, but there is a critical
difference. In dimensional metrology,
significant sources of uncertainty might
vary on a time scale of many years and
consequently it will require a very long
study to sample all likely sources of vari-
ation. For the study here, it is argued that
essentially all sources of variability can
be sampled in one day or at most a few
days, with the exception of solar storms.
As mentioned previously, solar storms or
satellite failures should be apparent if the
NIST archives are consulted. In addition,
a reasonable procedure for routine laser
calibrations would probably involve mul-
tiple 20-minute measurements over a
period of several hours; if an unexpect-
edly large Allan deviation of 20-minute
averages is observed, this could serve as
a warning that solar storms or other
measurement problems are increasing
the measurement uncertainty. A testing
period of several hours is sufficiently
long that, if setup problems such as
optical feedback were increasing the
measurement uncertainty, the increased
uncertainty will be reflected in an
increased Allan deviation of the measure-
ments. This should serve as a warning
that something might be wrong with the
measurement.
2.4 The Comb
There are potentially two types of errors
associated with the comb. First, it might
not have good short-term stability for a
variety of reasons. The stability studies
described above measure the combined
instability of the comb and other equip-
ment; the observed stability is satisfac-
tory for current needs and consequently
no more need be said regarding this
topic. The more critical question is:
Could there be systematic offsets in the
comb, even if there are no systematic
offsets in the GPSDO? This can occur if
there are errors in the locking of the rep-
etition frequency or the offset frequency
of the comb. To achieve 10
-12
relative
uncertainty in the measurement of an
optical frequency (5 10
14
Hz), the
20 MHz offset frequency cannot be in
error by more than 500 Hz, and the
100 MHz repetition frequency cannot be
in error by more than 10
-4
Hz.
For this system, one source of concern
is that signal-to-noise (S/N) levels in the
f 2f interferometer are often marginal.
Perhaps noise pulses, or some subtle
error in servo design, could shift the
offset frequency away from 20 MHz. The
accuracy of the offset frequency can be
checked by using an independent fre-
quency counter to measure the f 2f beat
frequency, although there is some danger
that noise might generate the same error
in the counter as in the electronics for the
servo system. Another test is to slightly
misalign the f 2f interferometer so as to
decrease the S/N ratio and verify that the
beat frequency between the comb and an
external laser does not change.
Figure 3. Beat frequency vs. time of day.
0 5 10 15 20 25 30
Time (hours)
27-Jul
28-Jul
29-Jul
30-Jul
31-Jul
1-Aug
B
e
a
t
F
r
e
q
u
e
n
c
y
(
M
H
z
)
13.6075
13.6070
13.6065
13.6060
13.6055
13.6050
13.6045
13.6040
TECHNICAL PAPERS
An additional test of the overall system performance, which
would probably reveal any conceivable failure in the control of
the offset frequency, is to compare measurements where the
offset frequency adds or subtracts from the measured beat fre-
quency between the comb and an external laser. This test simply
requires locking with the offset frequency alternately set to
20 MHz and to 20 MHz (reversed polarity on the servo output
of the f 2f feedback). If, for example, the beat frequency with
an external test laser changes by 40.002 MHz when the sign of
the offset frequency changes, this might indicate that the
20 MHz signal is in error by 1 kHz. Of course, it will be neces-
sary to switch the polarity multiple times and see if the appar-
ent error is statistically significant. Also note that the expected
change in frequency is only 40 MHz if possible changes in order
[changing n in equation (1)] are ignored, and in practice it may
be necessary to account for a change in order as well.
Note that if noise pulses or some more subtle error causes the
f 2f signal to be servoed to an incorrect value away from the
desired 20 MHz, it is not realistic to believe that the error will
switch signs when the servo polarity is reversed and thus hide
the error when performing this test. Although this test is some-
what cumbersome to carry out with good accuracy, it is a worth-
while test because consistent results will not be obtained unless
almost all elements of the measurement system are working
properly. The test cannot uncover an error in the repetition fre-
quency, but most of the other elements of the measurement
system are tested. Consistent results will not be obtained if there
is either a problem with the offset frequency servo or a problem
with counting the beat frequency between the comb and an
external laser.
The only other possible error of the comb would be that the
repetition frequency has an offset from the commanded value
supplied by the GPSDO. Suppose that there is a subtle error so
that when the comb is locked to a repetition frequency of
100 MHz (with the frequency derived from the GPSDO
10 MHz output), the error is small enough that it would not be
immediately obvious but large enough error to be of interest.
For example, imagine that the repetition frequency were
100 000 000.001 Hz, an error of 1 part in 10
11
. It is difficult to
imagine a plausible mechanism that would cause such a subtle
shift on a long-term basis. It is probably more characteristic of
a drifting phase than of noise-induced cycle slip in the phase
locked loop of the servo system. Perhaps temperature-depend-
ent phase shifts associated with some component of the system
(such as a zero-crossing detector) might give rise to slowly
varying phase errors for short periods of time, but the phase
shift could not continue indefinitely and would show up in the
stability tests. Actual miscounting of the repetition frequency or
cycle slip of a phase locked loop would cause much larger errors
that are easily observed. It thus appears to be highly unlikely
that the repetition frequency servo would be in error (other than
a complete failure to lock), but it is difficult to totally rule out
the possibility of errors without detailed knowledge of the servo
design.
A careful verification of the repetition frequency can provide
confidence in this critical element of the measurement, and it is
easy to directly verify that the repetition frequency faithfully
represents the GPSDO frequency, to within 1 part in 10
12
or
better, if a counter with 12 or more digits resolution is available.
This is done by using the GPSDO as an external timebase for a
counter. The counter timing resolution is 100 ps, which gives
1 part in 10
12
frequency resolution for a 100 s measurement.
It should be emphasized that all frequency counters used in
the experiments are referenced to the GPSDO, so that timebase
calibration is not needed. The uncertainty of the GPSDO fre-
quency is a separate issue that was discussed previously.
At this point it has been determined that the offset frequency
and repetition frequency servos are working properly. As these
are the only two parameters of the comb that affect the final
answer, it is now apparent that there are no significant errors
associated with the comb.
2.5 Beat Frequency Measurement
It must be verified that bad S/N ratio, electrical interference, or
similar effects are not degrading the measurement of the beat
frequency between the comb and the laser under test. A
common method for verifying proper performance is to misalign
the two interfering laser beams so as to decrease the S/N ratio
by 5 dB or 10 dB; if this does not change the measured beat fre-
quency then it is apparent that there are no difficulties in the
measurement. However, work is often done on the very edge of
acceptable S/N levels where this test is difficult to carry out in
a convincing manner. The authors are currently in the process
of implementing an alternative test that might provide real-time
verification of all aspects of the frequency counting. It should be
possible to simultaneously measure two beat signals, one at
some frequency f and the second, arising from interference of
the next comb order, at frequency f
rep
f. When the sum of the
two signals is exactly f
rep
, this should provide an excellent real-
time indicator that there are no problems with the beat fre-
quency measurement. The summation to f
rep
will only be exact
if the counters are gated simultaneously. To be a convincing test,
the summation need not be exact but should probably not differ
from f
rep
by more than a few percent of the desired uncertainty
of the measurement, because errors in the summation due to
miscounting associated with white noise tend to cancel when
using typical configurations for the electrical bandpass filters
that are used to separate the two signals. The cancellation is not
sufficiently exact to plausibly reduce the sensitivity of the test by
more than a factor of 10.
3. Summary: How to Achieve Confidence in Results
The uncertainty requirements needed for laser calibration are
very modest by the standards of an optical frequency comb;
there are no anticipated measurement needs exceeding 2 parts
in 10
12
. At this level, in situ tests can provide complete confi-
dence in the results. A summary of the argument is as follows:
a. The GPS signal which serves as the basic measurement
standard is traceable to NIST and calibrated daily. Very low
uncertainties can be claimed if NIST measurements indicate
that the GPS signal is operating with appropriate stability.
b. One form of internal consistency testing, observing the
short-term repeatability of measurements over a period of
several days, quantifies the combined uncertainties due to
TECHNICAL PAPERS
short-term fluctuations of all system components. (This is
true assuming that the manufacturer can vouch that steer-
ing algorithms do not make corrections with longer time
constants.) Furthermore, it is argued that these short-term
fluctuations fully quantify the uncertainty of measurement.
c. When the comb is operating properly (as can be determined
from internal consistency checks), and excluding statistical
uncertainties associated with short-term fluctuations
(already taken into account above), there are no known
sources of uncertainty associated with the comb itself that
could plausibly rise to a level that would be of interest to
this measurement (10
-13
).
d. When suitable S/N ratios are available, the only significant
sources of uncertainty in frequency counting of the beat
signal are either of a statistical nature, quantified by short-
term fluctuations, or drift of the timebase, which is not an
issue when using the GPSDO as a timebase for the counter.
Methods of verifying that the S/N ratio is sufficient have
been discussed.
No further discussion is needed to assess the uncertainty of
the measurement, and no ancillary equipment needs to be cali-
brated by an external laboratory in order to determine uncer-
tainty. The only calibration needs are fulfilled by the ongoing
monitoring of the GPS signal by NIST.
Separate from the question of uncertainty is the question of
design blunders in the equipment or measurement blunders by
an operator. These blunders could be of any magnitude. In prin-
ciple, software errors of a new model GPSDO might cause the
model to have a systematic, constant offsets relative to the SI
unit, so that the software cannot be trusted unless the product
line can make some claim of traceability to the SI second. (As
far as the authors are aware, no such constant offset has ever
been observed, but it is not impossible.) A more realistic
concern is that a GPSDO will have offsets that are only cor-
rected on a time scale of many days, but this problem can be
avoided if the manufacturer will publish sufficient information
so that it is possible to specify an appropriate testing period to
fully characterize the short-term fluctuations of the unit.
Any blunders associated with incorrect installation or com-
plete system failure of a GPSDO can be revealed by short-term
stability studies, which would not reproduce expected behavior
(such as, for example, the shape as seen in Fig. 2, which is char-
acteristic of the type of GPSDO used in this experiment.)
Operation of the optical frequency comb might be compro-
mised by a design blunder causing a long-term offset, which has
never been observed in practice but can be imagined. (For
example, if the comb had a faulty indicator for loss-of-lock, it
might operate with an arbitrary error. However, this would be
immediately obvious if the pulse repetition frequency is meas-
ured, and the stability would be so poor that it would probably
not even be possible to compare the comb to a second laser.)
Operation is more likely to be compromised by misalignment of
the f 2f interferometer with the resulting degradation of S/N
ratio. These kinds of errors are clearly beyond the realm of what
can be tested by sending equipment for calibration, but internal
consistency tests can be used to check for such errors on an
ongoing basis.
By far the most likely source of error in a laser calibration is
bad S/N in the beat between the comb and the test laser; this
can also be tested as was described previously.
Any of these tests can be repeated whenever there might be
a question if some error has occurred in the system, and some
of the tests can be implemented as part of the standard measure-
ment process and thus provide continuous quality assurance.
This is one great advantage of internal consistency tests relative
to other methods of verifying performance, including interlab-
oratory comparisons.
Below are five items that summarize these requirements and
could serve as a check-list for knowing that the uncertainty of
the system has been properly characterized:
1. It must be demonstrated that the short-term repeatability of
the system is consistent with uncertainty claims. Short-
term should probably not be less than two days and must
be longer than characteristic time constants associated with
the GPSDO software. Furthermore, if the measured short-
term stability is not consistent with what would be expected
for the particular model of GPSDO in use, then some other
evidence may be needed to show that the system is indeed
under GPS control and has not suffered a catastrophic
failure.
2. Additional evidence must be available to indicate that there
are no firmware errors or similar design blunders that
would cause a systematic frequency offset affecting the
entire GPSDO product line. Presumably at least one unit of
a particular model should be tested against a known stan-
dard with traceability to the SI second. Verification of the
integrity of the product line would logically be done by the
manufacturer rather than by individual users.
3. Internal consistency tests such as those described previ-
ously are needed to demonstrate proper operation of the
comb, competence of the operators, and correct measure-
ment of the beat frequency between the comb and a test
laser.
4. The tests that have been described here must be repeated
periodically to assure that everything continues to work
properly. Some tests, such as frequency measurements that
verify proper locking of the comb or checks that S/N levels
are satisfactory, can be built into a measurement process so
as to guarantee that every measurement is done properly.
Other tests, such as multi-day measurements of Allan devi-
ation, are cumbersome and might be repeated only annually
or even less frequently. Although full characterization of the
Allan deviation would be repeated only rarely, it is worth-
while to scrutinize the short-term repeatability of any
routine measurement to verify that observed frequency fluc-
tuations are not larger than what might normally be
expected. It is recommended that all measurements be
repeated over a period of at least a few hours; the observed
standard deviation of repeated measurements can be used
both to set a minimum plausible measurement uncertainty
and to alert the operator to potential measurement prob-
lems. Furthermore, for the greatest confidence in measure-
ment results, it would be worthwhile to reverse the 20 MHz
offset polarity at several points during the set of measure-
TECHNICAL PAPERS
ments and verify that the two polarities give consistent
answers.
5. To this list could be added additional considerations that
are not particularly unique to comb measurements but are
widely recognized as useful for assuring confidence in the
measurement process, such as, for example, assuring that
well-defined measurement procedures are adhered to.
Another example is that there should be some method of
verifying that there are no errors in the mathematical analy-
sis of results. In this regard it is noted that one should be
particularly cognizant of potential rounding errors when
analyzing a comb measurement, because of the very high
precision involved. Many mathematical errors will show up
in self-consistency testing, but certain calculation errors
will not be uncovered by the self-consistency tests.
4. Role of Interlaboratory Comparisons
There remains the question of the role of interlaboratory com-
parisons for establishing confidence in measurement results.
When the five requirements of Section 3 have been met, a com-
parison would add very little additional confidence in the meas-
urement process. It might still be argued that comparisons can
play a valuable role, because a comparison can reveal certain
types of blunders that cannot be discovered by other methods,
although a strict implementation of the five criteria of Section
3 leaves little room for unexpected errors. It is further noted
that standards, such as ASME B89.7.5-2006, encourage com-
parisons between independent laboratories [13], and, at the
international level, comparisons among national metrology
institutes (NMIs) are routinely expected to provide mutual con-
fidence in measurements. For combs, the most useful form of
such comparisons is not clear.
For current needs here at NIST, it would be desirable to verify
performance at a level where there is already full confidence in
the results, certainly below 2 10
-12
. In principle one could take
the comb to another NMI for a comparison, but this would be
difficult and is probably not a useful thing to do because it
would not test the full system, including GPSDO installation
issues that can only be tested locally. Even if one had a more
portable system based on a cesium clock, it is very difficult to
justify the expense required to directly compare two comb
systems. This comparison would require travel by a highly
trained operator, transport of a cesium frequency source, and
transport of the comb with its easily-misaligned optics. In sharp
contrast to the substantial required effort, it is again emphasized
that there is no known physical principle that would lead one to
doubt the validity of the measurements, and there is no known
blunder that would have eluded the self-consistency tests but
would be revealed by a comb comparison, with the possible
exception of calculation errors. It seems somewhat unlikely that
a comb comparison would ever uncover a measurement error
large enough to be of interest for a GPS-based system (greater
than 10
-14
) but small enough that it cannot be tested by simpler
procedures (less than 10
-10
).
For this reason comb comparisons may never be economically
justifiable. The money required for a comparison might be
better spent in carrying out more thorough self-consistency
testing whenever a measurement is performed. In short, instead
of testing for errors that have never been observed and are not
plausibly expected to occur, it is better to concentrate resources
on errors that are commonly observed, such as failing to check
the S/N ratio in the beat signal. (It might also be more cost-
effective to use the money for annual bonuses for technicians,
so as to improve morale!)
Nevertheless, direct comparisons of measurements are often
psychologically useful because they provide a strong motivation
for a laboratory to critically reevaluate procedures and carry out
needed periodic tests of the measurement system. Also, a com-
parison can uncover gross calculation blunders that might not
be evident from any other test. These benefits might be achieved
by using a stabilized laser as a transfer standard for comparing
comb systems indirectly.
One possible choice for a transfer standard is a 633 nm iodine
stabilized laser. These lasers are widely available in NMIs and
often serve as the top-level standard for dimensional metrology.
They are much easier to transport than a comb and might be
expected to reproduce frequency within a few kilohertz if oper-
ating parameters are readjusted following transport. Although
it is difficult to directly show agreement at the 10
-12
level when
using this procedure, it should be straightforward to demon-
strate agreement at a level somewhat below 10
-11
. When the five
requirements of Section 3 are fulfilled, it might be argued that
there is vanishingly small probability that an error lies in the
range between the 10
-11
level as validated by the comparison
and the 10
-12
or even the 10
-13
level that might be validated by
local self-consistency testing.
To give one example, the most common calculation blunders,
possibly a sign error, or possibly failure to include a frequency
offset from an acousto-optic modulator (AOM), are nearly four
orders of magnitude larger than 10
-11
. (Rounding errors repre-
sent a possible problem within this range, but there are much
more cost-efficient methods to assure that this trivial pitfall has
been avoided, and some types of rounding errors will show up
in self-consistency testing as well. Also, it must be noted that
comb-based systems that are more sophisticated than the one
used here may present more opportunities for small analysis
errors, and thus will require more careful scrutiny of the calcu-
lations.) At this time the authors have neither a plausible theo-
retical model nor direct experience indicating that errors in the
range between 10
-11
and 10
-13
can actually occur without
failure of the self-consistency tests. It is thus entirely reasonable
to claim a measurement uncertainty of 10
-12
or less, even if the
direct comparison evidence has uncertainties that are an order
of magnitude larger.
If one can accept the line of argument used above, then only
a small additional leap of imagination is required to accept
much more radical solutions to simplifying interlaboratory com-
parisons. Using an iodine stabilized laser as a transfer standard
is easier than a direct comparison of combs, but it is still a bur-
densome procedure, requiring transport of a delicate instrument
and travel by the operator. One possible simplification would be
to compare the comb to an iodine stabilized laser that is not
compared directly to a second comb but that can claim at least
a different traceability path to the international community.
TECHNICAL PAPERS
Even if the iodine stabilized laser is poorly maintained and char-
acterized, there should be little difficulty in using it to verify
comb performance with relative uncertainties at the 10
-10
level,
and a well-maintained laser will do much better than this. The
arguments regarding extension of confidence to the 10
-12
level
remain valid: if the measurement is verified at the 10
-10
level,
and if the five conditions are met, there is no reason to believe
that one cannot claim 10
-12
uncertainty. The comb being used
in Gaithersburg has been compared to an iodine stabilized laser
and found to be in good agreement with the expected frequency
of the laser.
In fact, there may be no compelling reason why this argument
cannot be continued through another two orders of magnitude.
The ultimate, radical, simplification would be to use a commer-
cial polarization or Zeeman stabilized laser as a transfer stan-
dard. Although it is more difficult to accept the idea that such
an imprecise laser might serve as a standard, there are several
significant advantages to using these lasers, and arguably there
are few disadvantages. Unlike iodine stabilized lasers, commer-
cial stabilized lasers are widely available, inexpensive instru-
ments that are sufficiently robust to be shipped without
difficulty and do not require travel of an operator. Certain com-
mercial stabilized lasers can be used reliably as transfer stan-
dards with expected drifts on the order of 10
-8
or below (if
minimal precautions are observed). This is an extremely cost-
effective alternative that greatly simplifies the comparison.
Although commercial lasers are typically several orders of
magnitude less stable than an iodine stabilized laser, one might
again argue that, when the requirements of Section 3 are satis-
fied, there are no plausible physical mechanisms that will cause
errors in the range between 10
-8
and 10
-13
, and consequently
there is no clear advantage to carrying out one of the more dif-
ficult comparisons described above. The polarization stabilized
laser would suffice to eliminate the possibility of calculation
errors or similar gross blunders as discussed previously, so it is
extremely likely that a comparison involving a polarization sta-
bilized laser would uncover any error that would be uncovered
by the more cumbersome test using an iodine stabilized laser or
using a traveling comb system.
Furthermore, one might even argue that in the real world,
comparison of commercial stabilized lasers provide better con-
fidence in results than does comparison of iodine stabilized
lasers, particularly for comparisons of laboratories that are not
NMIs. This comparison would demonstrate competence in
measuring a system with considerable commercial significance
and with somewhat different characteristics from an iodine sta-
bilized laser. For example, a Zeeman stabilized laser typically
has two output frequencies with different polarizations, and it
might be useful to sensitize testing laboratories to polarization-
related issues. It is also noted that commercial polarization sta-
bilized lasers can be easily modified so as to change the
frequency by more than 500 MHz, so that there is potential
ambiguity of the order n [see equation (1)] that must be
resolved by the testing laboratory, thus demonstrating a compe-
tence that is not challenged when comparing measurements of
iodine stabilized lasers. Realistically there may be more danger
of a laboratory making an error exceeding 1 part in 10
6
, due to
misidentification of order, than an error of 1 part in 10
12
due to
more subtle problems. (This is particularly true if a wavemeter
is used to establish order, and when working at frequencies
other than 633 nm, where laser standards are less commonly
available. Thus it might be best to use a polarization stabilized
laser that does not operate at 633 nm when carrying out the
comparison.)
In any event, there are economic and practical arguments
favoring the simplified procedure of using a commercial laser as
a transfer standard for laboratory intercomparisons. This situ-
ation arises because of the unique nature of the measurement:
(1) First, it is extremely difficult to carry out a comparison at an
uncertainty level small enough that it might plausibly help to
ally doubts regarding a realistic uncertainty budget that has not
been artificially inflated. (2) On the other hand, there is no clear
reason to doubt the ability to extrapolate wildly, using a very
coarse comparison to eliminate the possibility of blunders and
thus provide support for uncertainty claims that are four or five
orders of magnitude smaller than the comparison uncertainty!
In one sense the argument above is not as radical as it sounds.
A demanding comparison at the 10
-12
level is carried out, even
though it is not an interlaboratory comparison, when one com-
pares measurements with 20 MHz offset frequency and
20 MHz offset frequency. Similarly, when the repetition fre-
quency is measured with a frequency counter, one is effectively
comparing the expected behavior of the servo to an independ-
ent system (but sharing the GPSDO) measuring frequency at
the 10
-12
level. These tests clearly demonstrate the competence
of personnel to carry out measurements and correctly calculate
the answer at the 10
-12
level. What additional errors might be
uncovered by an extremely difficult interlaboratory comparison
at the 10
-12
level when a very simple self-consistency test has
already demonstrated competence at this level? All that the
interlaboratory comparison can add is (1) a test of the accuracy
of the GPSDO frequency, where one believes that the other tests
and procedures that have been described here will provide fool-
proof verification of the repetition frequency without the need
for a comparison and (2) a test for those calculation errors that
are common mode in the 20 MHz and 20 MHz measurements,
such as failure to include an AOM offset or certain rounding
errors. The only other likely source of error, misidentification of
the order, would not be tested by a comparison of iodine stabi-
lized lasers but could be tested using a simple commercial laser,
and this would also uncover gross mathematical errors such as
failure to include an AOM offset. All that remains are round-
ing errors. If rounding errors are deemed to be a significant
concern, this could always be tested via interlaboratory compar-
isons of analysis of some artificial data.
5. Conclusions
Through self-consistency testing and other means, it is possible
to verify a relative expanded uncertainty of less than 2 10
-12
for comb-based measurements. This is sufficient even for the
most demanding applications of dimensional metrology, such as
calibrating iodine stabilized lasers. It would appear to be
exceedingly unlikely that equipment calibration or participation
in an interlaboratory comparison would add value to the tests
TECHNICAL PAPERS
that have already been carried out. In the case of an interlabo-
ratory comparison, it is suggested that the value of a very simple
and inexpensive comparison may be comparable to the value of
the very difficult comparisons that would be required to directly
verify the claimed uncertainty. At present the authors are not
aware of any difficulty in employing a comparison with uncer-
tainty as large as a few parts in 10
8
to help justify a claimed
uncertainty of 10
-12
or below. This confidence in results that are
four or more orders of magnitude smaller than the comparison
uncertainty is probably unique to comb-based calibrations. It is
only justified because there is (at present) no plausible model of
how an error in the range 10
-8
to 10
-12
could arise without being
detected by internal consistency testing, and thus there is no
clear value in carrying out a more rigorous comparison.
A laboratory could easily participate in a comparison at the
10
-8
level and thus demonstrate their competence in measuring
lasers of commercial interest, even if the laboratory does not
have available an iodine stabilized laser to serve as the transfer
standard. At a later date, if the laboratory were asked to cali-
brate an iodine stabilized laser, they could presumably do so
with full confidence (although this would clearly invite careful
scrutiny of their documentation regarding self-consistency
testing). However, if the calibration showed an unrealistic error
in the iodine stabilized laser (exceeding perhaps 4 10
-10
), then
in all likelihood it is the comb rather than the laser that is in
error. If this were to occur, and if the five requirements of
Section 3 had been rigorously met, then the conclusions of this
paper would need serious reevaluation! (On the positive side,
the paper can then serve as a cautionary tale for future metrol-
ogists, an example of how one might be misled into thinking
that usual calibration chains or standard protocols for intercom-
parisons could be circumvented.) The strong confidence in the
internal testing procedures must be weighed against history:
there are many examples of measurements by highly competent
people where very subtle effects gave rise to errors far exceed-
ing the uncertainty claims. If comb technology and the technol-
ogy of portable frequency standards continues to improve, at
some point in the future it may become practical to directly
compare comb systems, in which case a direct comparison of
comb systems might become a more attractive method for ver-
ifying performance.
6. Acknowledgments
Thanks to Michael Lombardi of NIST for patiently answering
the authors questions regarding GPS and GPSDOs. Thanks
also to Lennart Robertsson of the Bureau International des
Poids et Mesures (BIPM) for useful discussions of comb systems
and how their uncertainty might be verified.
7. References
[1] Jun Ye and S.T. Cundiff, eds, Optical frequency combs and their
applications, Springer, New York, 2005.
[2] Th. Udem, R. Holzwarth, and T.W. Hnsch, Optical frequency
metrology, Nature (London), vol. 416, pp. 233237, 2002.
[3] The system is sold by Toptica Photonics AG. Some components
of the Toptica system were manufactured by Menlo Systems
GmbH.
[4] Certain commercial materials and equipment are identified in
order to specify adequately experimental procedures. In no case
does such identification imply recommendation or endorsement by
the National Institute of Standards and Technology, nor does it
imply that the items identified are necessarily the best available for
the purpose.
[5] The GPSDO is Timing Solutions model TSC 4410A.
[6] R.W. Fox, S.A. Diddams, A. Bartels, and L. Hollberg, Optical fre-
quency measurements with the global positioning system: tests
with an iodine-stabilized He-Ne laser, Appl. Opt., vol. 44,
pp.113120, 2005.
[7] L.-S. Ma, M. Zucco, S. Picard, L. Robertsson, and R.S. Windeler,
A new method to determine the absolute mode number of a
mode-locked femtosecond-laser comb used for absolute optical
frequency measurements, IEEE J. Sel. Top. Quantum Electron.,
vol. 9, pp. 10661071, 2003.
[8] L.-S. Ma, Z. Bi, A. Bartels, L. Robertsson, M. Zucco, R.S.
Windeler, G. Wilpers, C. Oates, L. Hollberg, and S.A. Diddams,
International comparisons of femtosecond laser frequency
combs, IEEE Trans. on Inst. and Meas., vol. 54, pp. 746749,
2005.
[9] See http://tf.nist.gov/service/gpscal.htm
[10] M.A. Lombardi, L.M. Nelson, A.N. Novick, and V.S. Zhang. Time
and frequency measurements using the global positioning system,
Cal Lab Magazine, Jul/Aug/Sept 2001, pp. 2933, 2001.
[11] M.A. Lombardi, A.N. Novick, and V.S. Zhang, Characterizing the
performance of GPS disciplined oscillators with respect to
UTC(NIST), Proc. of the 2005 IEEE Freq. Cont. Symp., pp.
677684, August 2005.
[12] D.A. Howe, The total deviation approach to long-term character-
ization of frequency stability, IEEE Trans. Ultrasonics, Ferro-
electrics, and Frequency Control, vol. 47, pp.11021110, 2000.
[13] ASME B89.7.5-2006, Metrological Traceability of Dimensional
Measurements to the SI Unit of Length, published by ASME, 3
Park Ave., New York, NY 10016.
With six regional labs, its easy to access Agilents calibration
services. And when we calibrate your instruments, you can be
sure that they retain their like new performance and accuracy.
Thorough, high quality calibrations start with automated,
factory-written procedures. All necessary adjustments are
included, no matter how many test points are out of caland
youll receive comprehensive pre- and post-adjustment data.
Well even advise you of the latest hardware and rmware
updates, clean parts that need it, and make minor repairs
all at no extra cost.
Turnaround time from your local cal center is normally ve
business days, and if youre within a two-hour radius, pick
up and delivery are free. Or when downtime is critical, our
Volume Onsite Calibration (VOSCAL) service brings Agilents
expertise to your site.
To learn more, contact us at service-info@agilent.com. Well
send an overview of your nearest cal centers capabilities and
a copy of our Remove all doubt brochure.
Agilent Technologies, Inc. 2007
www.agilent.com/nd/CAL_LAB
Let us pick up and deliveror calibrate your equipment at your site
Were in your neighborhood
Remove all doubt
Your instruments restored to
like-new condition, returned on time
Premier measurement expertise
NIST-traceable metrology
A2LA-accredited calibration available
For more information, send email to

service-info@agilent.com or call 1-800-829-4444
and well connect you to your local cal center:
Arlington Heights, IL
Bethlehem, PA
Chelmsford, MA
Durham, NC
Irvine, CA
Richardson, TX
TECHNICAL PAPERS
1. Introduction
One of the significant developments in coordinate metrology
over the last dozen or so years has been a growing interest in
evaluating the uncertainty of results produced by three dimen-
sional (3D) measuring systems, particularly coordinate measur-
ing machines (CMMs). Several motivations underlie this
developing concern. Principal drivers of this activity include the
steadily increasing employment of CMMs as tools for product
and process assessment, a generally increased concern for
product quality, greater globalization of trade, increased compe-
tition in the manufacturing environment and increasingly tighter
tolerances for manufactured goods. Two reflections of this inter-
est have been a growing body of research on the topic and the
recent appearance of national and international standards
dealing with or requiring the evaluation of measurement uncer-
tainty.
In this paper, the motivations and available methods for eval-
uating task-specific uncertainties (i.e. an uncertainty applicable
to a specific geometric dimensioning and tolerancing (GD&T)
parameter of a designated part feature, under particular condi-
tions of manufacture and measurement) in complex systems,
such as CMMs, are discussed, pointing out the strengths and
weaknesses of each. Then, simulation methods as applied to this
problem, with an emphasis on the necessary and desirable fea-
tures for a software application for CMM uncertainty evaluation
are presented. Next, an implementation of software that embod-
ies these features is described. Finally, a small set of application
Jon M. Baldwin, Kim D. Summerhays, Daniel A. Campbell and Richard P. Henke
Abstract: Uncertainty evaluations for coordinate measuring machine (CMM) metrology are problematic due to the
number, ranges, interactions and generally unknown sensitivity coefficients of the parameters that can influence the
measurement result. The situation is particularly difficult when a task-specific uncertainty is required and poses prob-
lems for both auditors and metrology practitioners. Auditors often lack satisfactory tools for a comprehensive assess-
ment of a clients claims of traceability. Measurement professionals, similarly, have difficulty demonstrating compliance
with measurement traceability requirements and, in addition, can find themselves at a real economic disadvantage if
reliable measurement uncertainties are not known. In this paper, the historical perspective of, the motivations for, and
the necessity of task-specific uncertainty evaluations are briefly discussed. This is followed by a presentation of the
requirements and desirable features of a credible method for task-specific CMM uncertainty evaluation. Next, a descrip-
tion of the major design features of a practical software application for evaluating uncertainties of CMM measurements
are presented. This is concluded by presenting several application examples and case studies which demonstrate that,
in the arena of task-specific CMM uncertainty evaluation, simulation methods exhibit notable strength and versatility.
Jon M. Baldwin
Kim D. Summerhays
Daniel A. Campbell
Richard P. Henke
MetroSage, LLC
26896 Shake Ridge Road
Volcano, CA 95689 USA
Email: jmbaldwin@metrosage.com
TECHNICAL PAPERS
examples and case studies are presented
that demonstrate the validity and utility
of simulation methods for task-specific
uncertainty evaluation of coordinate
measurement systems.
2. Task-Specific Uncertainty:
Historical Perspective and
Motivation
There has been an interest on the part of
researchers, going back at least to the
early 1990s [1], in methods to derive
task-specific measurement uncertainty
evaluations from more general CMM
performance parameters, and this effort
has since been carried forward by
workers in several countries. [2-6]
Generic CMM performance indices
have been available for some time, the
most prominent being those issued by
the International Organization for Stan-
dardization (ISO) [7] and by the Ameri-
can Society of Mechanical Engineers
(ASME). [8] While generic CMM per-
formance tests such as these are valuable
in comparative assessments of coordi-
nate measurement systems, they are
incapable of evaluating task-specific
measurement uncertainty.
Meanwhile, task-specific uncertainty
evaluation has become firmly established
as a necessary component process in
demonstrating measurement traceability
to national and international standards.
ISO 17025 [9] emphasizes the impor-
tance of uncertainty evaluation and,
most importantly, states that traceability
is achieved by means of an unbroken
chain of calibrations or comparisons
which include the measurement uncer-
tainty of each step. Similarly,
ANSI/NCSL Z540 [10] states, Calibra-
tion certificates and/or reports
shallstate the traceability tostan-
dards of measurement and shall provide
the measurement results and associated
uncertainty of measurement ASME
B89.7.5 [11] sets out in explicit detail
the requirements for traceability of
dimensional measurements.
The economic importance of uncer-
tainty evaluation has been further
emphasized in recent standards, notably
ISO 14253-1 [12] and ASME B89.7.3.1
[13], which create guidance for the for-
mulation of decision rules to govern the
acceptance or rejection of articles of
commerce. These standards provide for
possible economic penalties for greater
measurement uncertainty.
3. CMMs and Methods for
Evaluating Task-Specific
Uncertainty
The major factor that inspires the wide-
spread application of CMMs in industrial
dimensional metrology is their extreme
versatility; no other dimensional measur-
ing device is capable of determining such
a large variety of parameters on as large
a range of workpiece types and sizes.
Unfortunately, this same versatility leads
to difficulty when it is necessary to state
measurement uncertainty. In addition to
the many different measurands that are
evaluated in a typical CMM measure-
ment session, one is confronted with
almost unlimited sources of variability in
the conditions of measurement: work-
piece location and orientation, sensor
type(s) and configuration(s), environ-
ment, sampling strategy and computa-
tional considerations, to name just a few.
It is this tremendous variability that is at
the heart of the fact that typical CMM
calibrations and performance tests
cannot directly produce task-specific
measurement uncertainties.
This complexity can be further appre-
ciated by referring to Fig. 1, which shows
the CMM measurement traceability
chain in its entirety. Traceability is estab-
lished in a multi-step process, going all
the way back in an unbroken sequence to
national or international standards. Each
step contributes an uncertainty that must
be considered in developing the final,
task-specific uncertainty and traceability
statements. Some of these steps are rela-
tively straightforward. Some information
may be available to the CMM user with
little required effort. For example, the
uncertainties of the steps through the
artifact calibration are likely assessed by
national or international measurement
institutions and/or qualified calibration
sources, and should be captured on the
artifact calibration certificate. Others,
notably the CMM calibration and work-
piece measurement, are complicated and
are of necessity left in the hands of the
CMM user, who might not be well
equipped for their evaluation.
The available methods for CMM
uncertainty evaluation have been sum-
marized in a draft ISO technical report.
[14] They are:
Figure 1. Sources of uncertainty in the CMM traceability chain.
Atomic
Clock
CMM
Calibration
Workpiece
Measurement
and
Uncertainty
Statement
Measurement
Traceability
Iodine
Stabilized
Laser
Metrology
Laser
Artifact
Calibrator
A
B
8.500
32
3.458
Point Sampling
Strategy
CMM and
Probing Errors
Thermal
Errors
Part Form
Error
TECHNICAL PAPERS
1. Sensitivity Analysis, which involves listing each uncer-
tainty source, its magnitude, effect on the measurement
result and its correlation with other uncertainty sources,
then combining them in a manner that accounts for the
sensitivity factor of each source and the interactions
between sources. This is the approach described in the
ISO Guide to Uncertainty in Measurements (GUM) [15]
and is particularly useful if a mathematical model of the
measuring process can be had, because direct computation
of the sensitivity coefficients is possible.
2. Expert Judgment, which may be the only available method
if a mathematical model or measurement data are not
available. Its limitations in producing a defendable uncer-
tainty statement are evident.
3. Substitution, wherein repeated measurement of a cali-
brated master workpiece yields a range of errors and thus
the uncertainty. This is a powerful method of capturing the
relevant error sources and their interactions. Its major dis-
advantages are expense (need for multiple master parts)
and a reduction of the range of utility of the CMM.
4. Computer Simulation, where virtual models of the CMM,
the workpiece and the measurement process are created,
and estimates of the individual error sources are provided.
These data are then applied in repeated virtual measure-
ments. The distributions of the task-specific results yield
estimates of both bias and measurement variability and
hence uncertainty. Simulation methods are discussed in a
new supplement to the GUM. [16]
5. Measurement History, which is useful if large numbers of
measurements over time are available. This method can
place an upper bound on measurement uncertainty. It fails
to detect measurement bias.
Regardless of the method chosen to evaluate CMM uncer-
tainty, there are a few requirements for a credible method. A
minimum set of requirements is:
1. The chosen method must be comprehensive, i.e., all the
major influence variables must be considered.
2. All necessary GD&T parameters must be supported.
3. The evaluations of those parameters must conform to the
definitions established by the appropriate national and
international standards.
4. It must produce accurate and reliable results.
Several other qualities are highly desirable. They include:
1. The method should be versatile by supporting a useful
variety of CMM and probing system error models, and
workpiece and CMM thermal models.
2. It should demonstrate fidelity by allowing realistic con-
struction of measurement scenarios and metrology hard-
ware configurations, and correct choice of geometric
fitting algorithms.
3. It should be interoperable by accepting data from legacy
sources, e.g. existing workpiece designs and inspection
programs, and should provide a defined interface for com-
municating uncertainty information with other applica-
tions.
4. It should be flexible, offering a spectrum of tradeoffs
between cost of system characterization and quality of the
resulting uncertainty evaluations.
Table 1 presents a comparison of the five techniques as they
apply to evaluation of task-specific uncertainty of CMM meas-
urements, comparing them according to seven important prop-
erties.
Sensitivity analysis is rated questionable as regards tractabil-
ity and comprehensiveness, due to the need for explicit informa-
tion on the standard deviation and sensitivity factor for every
uncertainty source and on the correlation between every pair of
uncertainty sources. In some cases sensitivity coefficient calcu-
lation is impossible, since the measuring process cannot always
be analytically described. Sensitivity analysis is rated as poor in
regards to cost, due to the labor-intensive nature of the process,
Table 1. Uncertainty method scorecard for 3D metrology.
Sensitivity Analysis ????? ????? Strong Strong Weak Strong Weak
Expert Judgment Strong ????? ????? ????? Weak Strong ?????
Substitution Strong Strong Strong Strong Weak Weak Weak
Computer Simulation Strong ????? Strong Strong Strong Strong ?????
Measurement History Strong Strong Weak ????? Weak Weak Weak
Evaluation Method
T
r
a
c
t
a
b
l
e
C
o
m
p
r
e
h
e
n
s
i
v
e
D
e
t
e
c
t
s
M
e
a
s
u
r
e
m
e
n
t
B
i
a
s
D
e
t
e
c
t
s
M
e
a
s
u
r
e
m
e
n
t
V
a
r
i
a
b
i
l
i
t
y
V
e
r
s
a
t
i
l
e
P
r
e
d
i
c
t
i
v
e
E
c
o
n
o
m
i
c
a
l
Desireable Attributes of an Uncertainty Evaluation Method
TECHNICAL PAPERS
and poor from a versatility perspective
since much of the analysis is unique for
each individual application. Its strength
is that, properly conducted, sensitivity
analysis does a thorough job of detecting
both bias and variability.
The strength of expert judgment is its
tractability; it can be applied to any situ-
ation where the expert is confident.
Conversely, the comprehensiveness of
expert judgment is difficult to evaluate
and open to question. Cost can vary
widely. The versatility of the method is
not great; a separate consultation may be
required for each and every application.
The ability to detect bias and variability
is a strong function of the quality of the
expert; coverage of these issues is diffi-
cult to document and defend.
Substitution performs well except in
the areas of cost and versatility, which
are weaknesses of the method. Since cal-
ibration of the artifact is required and, in
typical CMM applications, a wide variety
of parts must be measured, this method
is generally uneconomical. Similarly, the
requirement of an artifact reasonably
similar, if not identical, to the workpiece
means that the method lacks versatility.
Measurement history is by definition
tractable and comprehensive. There are
ongoing costs associated with maintain-
ing and preserving the measurement
database. Its scope is limited to a partic-
ular workpiece. Measurement bias is
undetected. The method detects meas-
urement variability but cannot distin-
guish it from production process
variability.
Computer simulation is easily applied
to a wide range of problems; generally,
all the information required to set up a
simulation is available from the work-
piece and measurement process designs.
It can be variable in comprehensiveness;
the result is only as complete as the
model. This is primarily a concern to be
dealt with during initial selection of the
simulation software. Versatility and
tractability of simulation methods go
hand in hand; both depend on the same
information sources. Finally, simulation
methods easily capture both bias and
variability of the measurement process.
Simulation shares the strengths of sensi-
tivity analysis but often allows a more
complete assessment of interactions
between error sources.
4. Influence Quantities
on CMM Measurements
Earlier, it was mentioned that one of the
principal sources of difficulty in evaluat-
ing measurement variability is the
number and interactions of variables that
can affect a CMM measurement. It is
now necessary to visit this topic more
explicitly and in its full complexity.
CMM measurement influence quantities
can reasonably be categorized as illus-
trated in Fig. 2. Within this categoriza-
tion there are further levels of complexity
as detailed in Table 2. It is important to
note that this table may not be compre-
hensive, nor are all the influence vari-
ables listed likely to be important in
every instance. The significant point is
that in any CMM measurement the
sources of variation will be many and their
interactions will frequently be complex
and beyond the reach of analytical treat-
ment. It is likely that in most cases a few
of these errors will predominate.
Due to wide variations in measure-
ment systems, environments and meas-
urement objectives, and to the
prevalence of interactions between many
of these influence variables, it is not
practical to offer extensive generaliza-
tions concerning their relative impor-
tance. Thermal effects are commonly
significant, as can be uncompensated
geometric errors in the CMM. Dynamic
geometric effects become more signifi-
cant as measuring speed is increased.
Simple examples of the interaction
between workpiece form errors and sam-
pling strategies are well known. [17, 18]
5. Levels of CMM Uncertainty
It is necessary also to deal with the fact
that our knowledge of an uncertainty
source is often incomplete. This can be
illustrated by one example, CMM rigid
body geometric errors, frequently one of
the most important error sources. For
each axis, six functions of axis position
are needed: position error along the axis,
straightness in the two orthogonal direc-
tions, roll, pitch, and yaw, as well as
three scalar parameters, the out-of-
squareness values for each axis pair,
giving a total of 21 items in all for a three
axis Cartesian CMM. Introduction of a
rotary axis would require a similar set of
six functions plus two additional square-
ness parameters. The discussion and
examples cited in this paper will address
the most common case of three axis
Cartesian CMMs, but is extendable in
principle to other CMM geometries.
Figure 2. General categories of CMM measurement influence quantities.
CMM Errors
Environmental
Factors
Fitting Algorithms
Probe
System Erros
Feature
Form Errors
Sampling
Patterns
F
r
P = a
t
w
s
F
r
t
+ w
c

2
F
2
F
t
a
TECHNICAL PAPERS
While all these parameters can be
determined, the calibration process is
time consuming, on the order of a week
or more, and requires calibrated arti-
fact(s). Often, particularly in production
metrology facilities, much less is typically
known. For example, the commonly
used B89 test suite characterizes CMM
performance by six scalar parameters: a
repeatability parameter, three linear dis-
placement accuracy values, volumetric
performance and offset volumetric per-
formance. The only calibrated artifact
required is a length standard. Obviously,
the B89 results provide far too little
information to characterize fully the
rigid body errors of the CMM but the test
can be performed in about one to 1
days.
There is a continuum of tradeoff
choices available, where greater effort
and expense will produce higher quality
data and smaller estimates for the uncer-
tainties. It is reasonable to expect results
based on a full parametric characteriza-
tion to lie toward the high cost, higher
quality end of this range and that an
uncertainty evaluation based on a
simpler CMM performance test would
produce higher uncertainty values at
lower cost.
It should be noted that performance
test suites other than the B89 tests are
available and are sometimes cited by
CMM users and vendors. For example,
the ISO 10360-2 tests are commonly
cited by CMM vendors. Although the
system described here will produce
uncertainty evaluations based on a
variety of such tests, including all of the
above-mentioned options, the examples
and discussion in this paper will focus on
the B89 tests. While either of these test
suites offer substantial time saving as
compared to a full parametric character-
ization, it was observed that the B89
tests generally result in uncertainty esti-
mates that correspond more closely to
those derived from a full knowledge of
the rigid body errors. Generally, the
observed order of the uncertainty esti-
mates is full parametric specification <
B89 < ISO 10360.
The reasons observed for this are:
1. The B89 tests provide six parame-
ters to characterize CMM perform-
Table 2. Potential CMM measurement influence variables.
Influence
Category
Influence Factor Typical Source(s)
CMM Geometry
Rigid Body Errors
CMM Design/Construction,
Maintenance
Quasi-static Errors Workpiece Loading
Dynamic Errors
CMM Design,
Operating Parameters
Scale Resolution CMM Design
Sensor System
Probe Type
Availability,
Operator/Programmer Judgment
Stylus Configuration Operator/Programmer Judgement
Calibration Strategy
Control Software,
Stylus Bending Probe Selection
Approach Velocity Control Hardware
Probe Repeatability Probe Selection, Adjustment
Lobing Probe Design, Selection
Indexable Head
Repeatability
Design, Maintenance
Scanning Force and
Speed
Control Hardware,
Filtering Hardware, Software Design
Environment
Thermal Effects CMM, Workpiece
External Vibration Facility Design
Humidity Facility Design, Weather
Atmospheric Pressure Facility Design, Weather
Power,
Other Utility Variations
Facility Design
Lighting, Ventilation System Facility Design
Workpiece Factors
Systematic Form Error Manufacturing Method
Distortion by Fixturing Operator Practice
Sampling Strategy
Numbers and Locations of
Sampling Points
Data Analysis Fitting Algorithm Choice
Availability in Software,
TECHNICAL PAPERS
ance while the 10360 tests,
depending on how they are per-
formed, provide one, two or three
parameters. Thus the CMM model
is significantly more completely
determined by the B89 tests.
2. The B89 volumetric performance
test explicitly specifies the locations
and orientations in the CMM
working volume of the test artifact,
while the ISO 10360 protocol
leaves this matter to the users judg-
ment.
Whatever the basis for the CMM per-
formance evaluation, the result is an esti-
mate of the uncertainty of a single point
measurement and is an insufficient
metric for demonstrating traceability or
assessing conformance to a specification.
Ordinarily the value will vary through-
out the CMM working volume. At each
point in the working volume, there will
be a distribution of measured values.
Each time a point is measured some
value will be drawn from the distribution
of values for that region of the work
space. This value must be combined with
the other error sources to give the
overall point uncertainty and those point
uncertainties propagated through the
measurement process to give a complete
uncertainty statement.
6. Description of the Method
At the heart of our implementation is a
National Institute of Standards and Tech-
nology (NIST) developed method called
Simulation by Constraints (SBC). [3]
The choice of method was based largely
on the desire for flexibility to the user in
cost/benefit tradeoffs, as mentioned
earlier. SBC provides this by allowing
simulations to be set up and executed in
the face of incomplete information. This
can be seen by reference to Fig. 3 where,
for the purpose of illustration, it focuses
on just one aspect of CMM uncertainty
evaluation, the effect of rigid body
mechanical errors. The method begins
with the recognition that the information
available to define the uncertainty source
may be incomplete; in this case, using a
CMM performance test that does not
completely define the CMM geometry.
For example, there usually will be many
sets of 21 rigid body parameters that
would result in the same discovered set
of six B89 parameters, the Bounding
Measurement Set (BMS). The SBC
method would begin with the generation
of an adequate number (typically hun-
dreds) of rigid body parameter sets that
result in B89 numbers near the BMS
values. Each of these sets of 21 parame-
ters (3 scalars and 18 functions) can be
thought of as a virtual CMM. For each
virtual CMM the error of each individual
point measured on the workpiece is com-
puted. These points (with their errors)
are submitted to the CMM data process-
ing algorithms to obtain the correspon-
ding substitute geometries for all the
measurement features of concern. The
substitute geometries are used to evalu-
ate all the GD&T parameters of interest
and the bias and range of the distribution
of the results for each parameter pro-
vides its measurement uncertainty. The
extension of the SBC concept to other
error sources is straightforward.
7. System Architecture
The software architecture that was devel-
oped for task-specific uncertainty evalu-
ation is shown in Fig. 4. One of the
objectives of this architecture is to lever-
age, as much as possible, commercial, off
the shelf software capabilities. An imple-
mentation of this architecture
(PUNDIT/CMM) has been created.
Central to the implementation is the def-
inition of the workpiece, represented in
the context of a 3D geometric modeling
kernel. This kernel supplies geometric
modeling services. The current imple-
mentation is based on the ACIS solid
modeler,
1
although another geometry
kernel could be substituted. Surrounding
this kernel is the dimensioning and toler-
ancing layer which provides the essential
services of identification, assignment
and checking of tolerance features,
datum reference frames and tolerances.
This functionality is provided by
FBTol.
2
The services provided by the
dimensioning and tolerancing layer fulfill
many of the metrology-related functions
currently neglected or inadequately pro-
vided by all known computer-aided
design (CAD) geometry kernels; specifi-
cally the abilities to associate the raw
geometry with the workpiece features to
be toleranced and measured, to unam-
biguously assign tolerances to those fea-
tures, and to create and associate datum
reference frames. Finally, the outermost
Measurement Data
forming Bounding
Measurement Set
One-to-many mapping via
Data fitting to Kinematic Eqns
Population of Possible
Substitute Geometry
Errors is Expressed as
Measurand Uncertainty
One-to-one mapping via
Fitting algorithm for
Substitute Geometry
One-to-one mapping via
Kinematic Eqns
Population of
good virtual
CMM states
Point Coordinate
Errors
Substitute Geometry
Errors
Substitute Geometry
Uncertainty
Figure 3. Principles of uncertainty evaluation by simulation by constraints, using CMM
kinematic errors as an illustration.
e.g. B89.4.1 data
1
ACIS is a registered trademark of Spatial
Technologies, Inc.
2
FBTol is a registered trademark of Honey-
well Federal Manufacturing & Technologies.
TECHNICAL PAPERS
software, or interface and special services, layer provides CMM
metrology-specific functions, such as the simulation engine, the
geometric fitting and tolerance evaluation algorithms, the
models of the various components of the CMM metrology
process, and the user interface which allows parameterization
of the measurement process.
The operation of this architecture in a typical simulation run
might be as follows. As mentioned above, the outer software
(interface and services) layer provides most user interface func-
tions; for conciseness, this involvement is not always specifically
identified. A workpiece model is imported by the interface layer
and is passed to the geometry kernel. The dimensioning and tol-
erancing layer operates on the model, generally using a combi-
nation of tolerancing wizards and user interaction, to identify
tolerance features, supply datum information and tolerances.
This is an essential first step, after which most of the remaining
operations may be performed in any convenient sequence. The
interface and services layer creates the CMM and probing
system models, and the environment model. The dimensioning
and tolerancing layer provides services for creating the measure-
ment plan, i.e., how the features are to be probed, fitting algo-
rithm selections, etc. The interface and services layer is used to
supply the manufacturing information, i.e., the shape and ampli-
tude of the form errors for each feature. Finally, the interface
and services layer provides the simulation engine, drives the
simulation run, and presents the results.
8. User Interface Design
The user interface is crucial to the usability of the uncertainty
evaluation system, in that a careful and logical presentation of
the measurement process greatly facilitates straightforward and
natural-seeming development of measurement scenarios. At the
highest level, the user interface is organized into seven
tabbed activities, each representing a distinct aspect of the
measurement process. These tabs are presented at the bottom of
the display screen (see Fig. 5) and, with the exception of the
workpiece tab, may be visited in any desired order.
Examples of each of the tabbed pages are presented in Fig. 6.
From left to right and top to bottom, these are:
a. The Workpiece Tab. Here, the as-designed workpiece
geometry and the associated tolerance requirements are
defined. Generally, the workpiece model will be created in
and, imported from, an independent CAD system,
although a rudimentary facility, suitable for modeling
workpieces of simple geometry, is provided. A graphical
view of the model occupies the right-hand pane of the tab.
Once the model has been imported or created, tolerancing
is applied. This may be done automatically using built-in
software wizards, interactively by the operator, or with a
combination of these methods. If a legacy inspection
program is available in the Dimensional Measuring Inter-
face Standard (DMIS) [19] format, the feature, datum and
tolerance information can be extracted and automatically
imported. In any case the tolerance information, including
tolerance features, datum reference frames, material con-
dition modifiers and applied tolerances, is displayed in
lists in the left-hand pane. Also in this tab, the tolerance
scheme applied to the workpiece can be verified. A suite
of tools is available that can be used to automatically
determine if any aspect of the part design is over or under
constrained and if tolerance definition is complete. This
capability, although not essential for uncertainty evalua-
tion, has value from the very beginning of the workpiece
design and tends to encourage concurrent development of
the design and the measurement.
b. The Manufacturing Information Tab. The previous tab
was concerned with the part as designed; this one deals
with the part as manufactured. Specifically, it accounts for
the fact that no manufacturing process creates features of
ideal shape. Many of the shape errors are systematic and
characteristic of the manufacturing methods and parame-
ters. [20] It is well established that the interaction of shape
(form) errors and the sampling pattern used in the CMM
measurement can be a significant source of uncertainty.
[21] PUNDIT/CMM has available several ways of apply-
ing form errors to each feature; one of them is shown in
Fig. 6. The available methods are:
1. User Query, where someone sufficiently knowledgeable
about the manufacturing process can apply combina-
tions of specific functional shapes, e.g. lobing, taper,
bellmouth, twist, etc., and random error.
2. Dense Data, where, if one or more samples of the
actual production have been carefully and completely
measured, the discovered form errors can be applied in
the uncertainty evaluation.
3. Manufacturing Process, where a library of form errors
can be assembled and reused as needed.
It is also worth noting the pair of radio buttons near the
top center of the screen. These allow perfect workpiece
form to be temporarily applied, and are useful in doing
what if? types of analysis, in this case determining how
Figure 4. Uncertainty evaluation system architecture.
Results
Features Tolerancing
Datum Reference
Frames
3D
Geometric
Modeler
Simulation
Engine
Fitting
Algorithms
CMM
Model
Probe
Model
Mfg.
Errors
Sampling
Patterns
Environmental
Factors
TECHNICAL PAPERS
much form error contributes to the
total uncertainty. Similar capabili-
ties will be seen for other influence
quantities.
c. The CMM Tab. Here, the CMM
model is defined. The CMM design
(e.g. bridge, cantilever, etc.) and
axis stacking are designated, along
with the working volume and home
location. A CMM performance
model is chosen. Currently avail-
able are perfect machine, B89
model, full parametric model, the
ISO 10360 model and an extended
version of the latter, currently
under consideration for ISO adop-
tion. A database of CMM perform-
ance parameters is provided and
initially is populated with selected
manufacturer-published informa-
tion. The database can be updated
to include new machines and user-
determined performance parame-
ters for specific CMMs. There is
also a dialog for entering measuring
velocities, accelerations, etc. These
are required when the transient
thermal model is invoked, as will
be described in a following section.
d. The Probe Tab. PUNDIT/CMM
currently accommodates contact
probes. There are models for
switching and piezo probes, as well
as a perfect probe option. Allowed
probe configurations are fixed
single tip, fixed multitip, and artic-
ulated single tip. The probe per-
formance test is also chosen here,
the options being the ASME
B89.4.1, ISO 10360, and
VDI/VDE [22] tests as well as an
extended version of the ISO tests.
e. The Environment Tab. Thermal
effects are almost always the pre-
dominant environmental source of
error in CMM measurements. Two
basic thermal models are sup-
ported: a static model where tem-
perature is constant throughout
the measurement and one which
allows the workpiece temperature
to change in the course of the meas-
urement. Within each of these
models, a selection is available that
allows for several levels of temper-
ature compensation: none, com-
pensation for CMM temperature
only, compensation for CMM and
workpiece temperature where both
are assumed to be at the same tem-
perature, and full compensation
when part and CMM may be at dif-
ferent temperatures. In recognition
of the fact that some highly accu-
rate CMMs employ laser interfer-
ometry to determine CMM
position, environmental effects on
interferometric scales can be
modeled, as well.
Figure 5. Highest level user interface.
Figure 6. User interface illustrations.
a. Workpiece Tab b. Manufacturing Information Tab
c. CMM Tab d. Probe Tab
e. Environment Tab
g. Results Tab
f. Measurement Plan Tab
CMM Probe Environment Measurement Plan Manufacturing Info Results Workpiece
f. The Measurement Plan Tab. In this tab, the numbers and
distributions of measurement points, the order of meas-
urement, the position of the workpiece in the CMM meas-
urement volume, the fitting algorithm and the probe
selection are specified. Point placement on a feature can
be specified in a variety of ways: points can be placed man-
ually, a variety of regular patterns can be applied by auto-
mated routines, and optimized patterns [21] designed to
give the best results in the face of feature form errors can
be imported as can point patterns from existing DMIS
measurement programs. Edge offsets can be specified for
the automated pattern routines, and points falling into
voids on the model are automatically rejected. If needed,
DMIS code for measurement of each feature can be pro-
duced here.
g. The Results Tab. Here, the uncertainty evaluation analy-
sis is conducted and the results displayed. Along the left
side of the window is a tree that lists the tolerances that
have been applied and the features to which they belong.
To the right is a pane which will display a histogram of the
errors for whatever feature/tolerance combination is
chosen on the left. Along the bottom are controls for
selecting the number of simulations to be run, and display-
ing the progress of the run. For a part of moderate com-
plexity, a series of a few hundred simulations can be run
in 60 seconds or so. All of the feature/tolerance pairs are
analyzed in each simulation run, so all the results are
immediately viewable by selecting the appropriate combi-
nation from the tree. A text output of the analysis and
screen grabs of the histograms are also available.
9. Applications and Case Studies
Presented here are some simple but useful application case
studies produced with the uncertainty evaluation system.
9.1 Validation of the Method
Simulation software for evaluating CMM task-specific measure-
ment uncertainty is complex and comprehensive testing for val-
idation is similarly so. Validation methodologies have been the
subject of recent research [23] and have begun to be codified.
[24] Considered here are some examples of these methods as
applied to PUNDT/CMM.
9.1.1 Physical Measurements
The effectiveness of uncertainty evaluation software can be
gauged by showing that for calibrated parts, the observed meas-
urement errors are reasonably bounded by the computed uncer-
tainties. A study of this sort, using PUNDIT/CMM, was
conducted by metrologists at the National Institute of Standards
and Technology and at the Oak Ridge Metrology Center, using
various artifacts and CMMs. [23] A typical example from that
study is presented here. A calibrated 300-mm diameter disk was
positioned in a variety of locations and orientations in the meas-
uring volume of a high accuracy CMM and, using a fixed sam-
pling pattern, measured for diameter and circularity thus
providing a set of known measurement errors. The B89.4.1 per-
formance test suite and ISO probe tests were also conducted on
this same CMM and used to define the CMM performance to
PUNDIT/CMM. Some of the results are presented in Fig. 7. The
errors at all positions and locations are bounded by the uncer-
tainties predicted by the software. It is not surprising that the
uncertainties provide somewhat loose bounds, considering that
the CMM performance was specified by only the six B89 param-
eters. Better information on the CMM, e.g. a full parametric
specification of the rigid body errors, would be expected to yield
a tighter bounding, as can be illustrated using virtual calibrated
artifacts. An example is presented in Figs. 8 and 9. Figure 8
shows the part and the set of five representative GD&T param-
eters used for comparison. Figure 9 compares the full paramet-
ric specification results (FPS) to the corresponding values
deduced from only (B89) performance test data. The calcula-
tions here were based on deactivating all error sources other
than the CMM itself. The simulation results based on B89 per-
formance testing bound those based on the full parametric spec-
ification by the same order as they bound the actual
measurement results in Fig. 7.
TECHNICAL PAPERS
Diameter
Artifact position (arbitrary) Artifact position (arbitrary)
3.5
3
2.5
2
1.5
1
1 2 3 4 5
Circularity
7
6
5
4
3
2
1
0
1 2 3 4 5
U
n
c
e
r
t
a
i
n
t
y
o
r
E
r
r
o
r
,
m
U
n
c
e
r
t
a
i
n
t
y
o
r
E
r
r
o
r
,
m
Figure 7. Comparison of simulated uncertainty and measurement error.
x
x
x x
x
x
x
x
x
x
Simulated uncertainty Measured error x
TECHNICAL PAPERS
9.1.2 Reference Values and
Component Testing
A reference value is a special case result
that has a known value, which can be
tested for in simulation. PUNDIT/CMM
incorporates two features that allow
component testing for reference values
within the integrated software environ-
ment. The first of these is the fact that a
model of the geometrically-perfect work-
piece of interest is available. This means
that, in effect, the software can be tested
using a nearly infinite set of virtual cali-
brated parts. The second feature is the
ability to temporarily deactivate entire
classes of influence quantity. This allows
a focus on particular contributors to
uncertainty, where it is relatively easy to
develop test cases and to verify that com-
puted results are compatible with antic-
ipations. Described here are just two
examples, one nearly trivial and the other
somewhat more complicated, both
involving use of reference values.
In the first instance a 500-mm step
gage part was measured by simulation
under conditions where all error sources
other than thermal ones were turned off.
The step gage was stipulated to have a
coefficient of thermal expansion of pre-
cisely 10 ppm/C and a temperature of
exactly 21 C. The simulations showed
systematic length errors of 1 m at
100 mm, 2 m at 200 mm, etc., just as
anticipated.
As a second example of focus on spe-
cific error sources, consider a 10 mm
inside diameter cylinder sampled at three
equiangular points at each of three levels
along its length (the points at each level
eclipse those of other levels when the
cylinder is viewed end-on). The cylinder
surface was specified to have a 3-lobe
(sinusoidal) form error (i.e., cylindricity)
of peak-to-peak amplitude 10m. The
CMM, probe and thermal conditions are
set to perfect. PUNDIT/CMM then
predicts a cylindricity bias (mean error in
measurement) of the (negative) full
amplitude ( 10 m) and no variability
around this. To see why this is so, note
that in simulation, the phasing of the
form error to that of the sampling
pattern is randomized on each cycle. But
because the form error here is entirely
systematic, and the number of sample
Figure 8. Workpiece and tolerances used in case study discussed in Section 9.1.1.
Figure 9. Relative estimated uncertainties for FPS and B89 CMM models.
0.06 D 0.08 A D B
0.04 A 25.00 0.02 0.06 A D B
FPS
CONC PROF ANGR POS DIAM
U
n
c
e
r
t
a
i
n
t
y
(
m
i
c
r
o
n
s
)
B89
5
4
3
2
1
0
Figure 10. Distribution of diameter errors for cylinder with three lobes, sampled at three
equiangular positions at each of three levels.
10.000dHole Diameter
Mean Error = 0.000474 mm Std Dev = 0.006763 mm
0
.
0
0
8
0
.
0
0
6
0
.
0
0
4
0
.
0
0
2
0
.
0
0
0
0
.
0
0
2
0
.
0
0
4
0
.
0
0
6
0
.
0
0
8
P
r
o
b
a
b
i
l
i
t
y
o
f
E
r
r
o
r
Error
TECHNICAL PAPERS
points at each level is equal to the number of lobes, the cylin-
dricity obtained in the simulated measurements is always zero
regardless of the phasing. Thus on each cycle, the bias [= (mea-
sured cylindricity) (true cylindricity)] is the full 10m.
Whatever the phasing of the lobing to the points, the measured
cylinder center location should remain fixed and
PUNDIT/CMM predicts zero bias and zero variability in meas-
ured position. Finally, the measured cylinder radius can be
expected to range above and below its nominal value by half of
the cylindricity, depending upon the phasing of the lobing to the
sampling. The measured diameter would thus range over the full
cylindricity (10 m). Furthermore, the distribution of the diam-
eters can be expected to be quite non-Gaussian, since the sinu-
soidal lobing form error yields a notably higher sampling rate at
its extrema than at intermediate values. This is just what
PUNDIT/CMM shows in its error histogram (Fig. 10). Several
of the examples that follow may be considered as instances of
validation by component testing, as well as demonstrations of
the importance of specific error sources.
9.2 Relative Importance of Error Sources
This example illustrates the relative contributions of several
error sources to overall uncertainty. The effects of CMM geom-
etry errors, probe errors and thermal nonidealities were consid-
ered. The CMM linear accuracies were 3.0 m, 2.1 m and 2.5
m for the x, y and z axes, respectively. The volumetric perform-
ance was 7.2 m. Offset volumetric performance was 7.1 m/m
and the repeatability was 1 m. The piezoelectric probe was
modeled with a random error () of 5 m. The CMM scales
were considered to be temperature insensitive; the workpiece
was aluminum, temperature compensated with an expansion
coefficient of 22 2 ppm/C and a temperature of 25 3 C.
There was no form error added to either the measured cylindri-
cal feature nor to the datum features, and the datum features
were assumed to be perfectly measured. These last points are
important since, generally, errors in the datum features will
propagate into the uncertainty of the position and orientation of
the feature under consideration.
The computed uncertainty is shown separately in Fig. 11 for
size, location and form, and for every possible combination of
error sources. The notation 101 signifies, for example, that
CMM and thermal errors were considered in that particular
experiment, but not probe error. In each case 300 simulated
inspections were performed. While the interrelationships of the
error sources were not treated explicitly, their relative independ-
ence can be seen from the fact that the uncertainty values from
treatment of two error sources at once are approximately equal
to the root-sum-of-squares of the uncertainties from the error
sources treated singly.
9.3 Form Error/Sampling Pattern Interactions
This example illustrates the interaction between feature form
errors and sampling strategy. Again, the measurement of a cylin-
drical hole has been simulated, but it is nowassumed that the part
is measured with a perfect CMM and probe and that there are no
thermal effects. A three lobe form error of 0.4 m amplitude was
assumed in one experiment and a combination of three lobe error
and random error, each of 0.2 m maximum amplitude was used
in another. The sampling patterns all used even numbers of points
arranged at two levels near the ends of the cylinder and evenly
spaced at each level. The eclipsed patterns (labeled E in Fig.
12) have the points at each level placed at the same angular posi-
tions while in the staggered patterns (labeled S) the point
positions on one level are rotated by half the angular spacing rel-
ative to the other level. Notice the oscillatory behavior with
maxima when the number of points at each level is an integer
multiple of the lobing frequency. The staggered pattern helps to
damp the effect but even then it persists to fairly large numbers
of sampled points and can be seen even in the mixed cases
where the random error is as large as the systematic.
9.4 Effect of Errors in Datum Measurement
It is generally understood that errors in the measurement of
datum surfaces will be reflected in measurements made relative
to those datums. CMM users may appreciate less well the mag-
nitude of the effect. This example provides a simple illustration.
The workpiece is a rectangular block; the feature of interest is
a cylindrical hole centrally located in one face of the block. The
Figure 11. Contributions of several influence variables to overall uncertainty.
Size
Cylindricity
000 001 010 011 100 101 110 111
U
n
c
e
r
t
a
i
n
t
y
(
m
i
c
r
o
n
s
)
Factor Combinations (CMM-Probe-Thermal)
Position
0.035
0.030
0.025
0.020
0.015
0.010
0.005
0
TECHNICAL PAPERS
hole is toleranced positionally relative to
three planar datum surfaces, each of
which is considered to have been manu-
factured with 100 m of random form
error. All else about the measurement is
considered perfect. Figure 13 shows the
relation of positional uncertainty of the
hole as a function of sampling density of
the datum surfaces. Suppose, for
example, that the hole is assigned a posi-
tional tolerance of 0.5 mm. Remember-
ing that in a real measurement there will
be other sources of error, it might be con-
sidered wise to keep the error from this
particular source to 10 % or less of the
tolerance. This requires a minimum of
about 250 probing points per datum
surface.
10. Conclusions
The design of a comprehensive system
for evaluating the task-specific uncer-
tainty of measurements made with
CMMs has been described, and examples
of its application have been shown. With
replacement of some of the CMM-spe-
cific models, it could be adapted to other
3D metrology systems, for example,
articulated arm CMMs and laser theodo-
lites. The software has been shown to be
robust and versatile. It can be of use both
to auditors and measurement profession-
als when demonstrated traceability of
CMM results is required. Measurement
practitioners will also find the system of
value in identifying and reducing the
sources of uncertainty in their measure-
ments, resulting in economic benefit.
The software also can be beneficial in a
variety of subsidiary functions, includ-
ing:
a. Verifying that tolerance applica-
tions are complete, consistent and
unambiguous;
b. Making the right choice when pur-
chasing a CMM;
c. Finding and fixing the weak link
in a CMM system;
d. Choosing the best CMM for a spe-
cific job; and
e. Training users in proper CMM
measuring procedures.
11. References
[1] E. Trapet, F. Wldele, and U. Wiegand,
Coordinate Measuring Machines in the
Calibration Chain, CNAM (Conserva-
toire National des Arts et Mtiers),
Paris, 1994.
[2] E. Trapet and F. Wldele, The virtual
CMM concept, Advanced Mathematical
Tools in Metrology II., P. Ciarlini, M. G.
Cox, F. Pavese and D. Richter, eds., Sin-
gapore, World Scientific, vol. 40, pp.
238247, 1995.
[3] S.D. Phillips, B.R. Borchardt., et al.,
The Calculation of CMM Measurement
Uncertainty via The Method of Simula-
tion by Constraints, Proceedings of the
12th Annual Meeting of the American
Society for Precision Engineering,
Norfolk, VA, pp. 443446, October
1997.
[4] A. Weckenmann and M. Knauer, The
influence of measurement strategy on
the uncertainty of CMM-measure-
ments, Annals of the CIRP (College
International pour la Recherche en Pro-
ductique), vol. 47, no. 1, pp. 451454,
1998.
[5] A. Balsamo, M. Di Ciommo, R. Mugno,
B.I. Rebaglia, E. Ricci, and R. Grella,
Evaluation of CMM Uncertainty
through Monte Carlo simulations,
Annals of the CIRP, vol. 48, no. 1, pp.
4258, 1999.
[6] B. Van Dorp, H. Haitjema, et al.,
Virtual CMM using Monte Carlo
methods based on frequency content of
the error signal, Proceedings of the
SPIE, vol. 4401, pp. 158167, 2001.
[7] Geometrical product specifications
(GPS)Acceptance and reverification
tests for coordinate measuring machines
(CMM), ISO 10360 Parts 15, Interna-
tional Organization for Standardization,
Geneva, 2000.
[8] Methods for Performance Evaluation of
Coordinate Measuring Machines,
ASME B89.4.1, American Society of
Mechanical Engineers, New York, 1997.
Figure 12. Interaction of sampling strategy with feature form.
Figure 13. Effect of datum form error on probe point density required to achieve a speci-
fied positional uncertainty for a dependant feature.
# of Sample Points
U
n
c
e
r
t
a
i
n
t
y
(
m
m
)
0 5 10 15 20 25 30
0.0008
0.0006
0.0004
0.0002
0
# of Sample Points
U
n
c
e
r
t
a
i
n
t
y
(
m
m
)
0 200 400 600 800 1000 1200
0.3
0.25
0.2
0.15
0.1
0.05
0
3-Lobe E
3-Lobe S
Mixed E
Mixed S
TECHNICAL PAPERS
[9] General Requirements for the Competence of Testing and Cali-
bration Laboratories, ISO 17025, International Organization for
Standardization, Geneva, 1999.
[10] American National Standard for CalibrationCalibration Labora-
tories and Measuring and Test EquipmentGeneral Require-
ments, ANSI/NCSL Z5401, National Conference of Standards
Laboratories, Boulder, CO, 1994.
[11] Metrological Traceability of Dimensional Measurements to the SI
Unit of Length, ASME B89.7.5, American Society of Mechanical
Engineers, New York, 2006.
[12] Geometrical Product Specifications (GPS)Inspection by Meas-
urement of Workpieces and Measuring EquipmentPart 1: Deci-
sion Rules for Proving Conformance or Non-Conformance with
Specifications, ISO 142531, First Edition, Geneva, 1998.
[13] Guidelines for Decision Rules: Considering Measurement Uncer-
tainty in Determining Conformance to Specifications, ASME
B89.7.3.12001, American Society of Mechanical Engineers, New
York, 2002.
[14] Techniques of Determining the Uncertainty of Measurement in
Coordinate Metrology, ISO Technical Report 155301 (draft),
July 16, 1998.
[15] Guide to the Expression of Uncertainty in Measurement, ISO
Guide 98, Geneva, 1995.
[16] Guide to the Expression of Uncertainty in Measurement, Supple-
ment 1, Propagation of distributions using a Monte Carlo
method, ISO Guide 98 Supplement 1, (final draft), Geneva, 2006.
[17] S.D. Phillips, Performance Evaluations, Coordinate Measuring
Machines and Systems, J. Bosch, ed., Marcel Dekker, 1995.
[18] R. Edgeworth, R. Wilhelm, Characterization of workpiece form
uncertainty for uniform sampling patterns, North American Man-
ufacturing Research Conference, New York, May 2005.
[19] Dimensional Measuring Interface Standard, ANSI, Revision 5.0,
CAM-I 105.02004, Bedford, TX, 2004.
[20] R.P. Henke, K. D. Summerhays, J.M. Baldwin, R.M. Cassou, and
C.W. Brown, Methods for evaluation of systematic geometric
deviations in machined parts and their relation to process vari-
ables, Precision Engineering, vol. 23, no. 4, pp. 273292, 1999.
[21] K.D. Summerhays, R. P. Henke, J.M. Baldwin, R.M. Cassou, and
C.W. Brown, Optimizing discrete point sample patterns and
measurement data analysis on internal cylindrical surfaces with
systematic form deviations, Precision Engineering, vol. 26, no. 1,
pp. 105121, 2002.
[22] Accuracy of coordinate measuring machines: characteristic
parameters and their checking, VDI/VDE 2617, part 2.1, Dussel-
dorf, Germany, 1986.
[23] S.D. Phillips, B. Borchardt, A.J. Abackerli, et al., The Validation
of CMM Task Specific Measurement Uncertainty Software, Pro-
ceedings of the American Society for Precision Engineering
Summer Topical Meeting on Coordinate Measuring Machines,
Charlotte, NC, June 2003.
[24] Geometric Product Specification (GPS)Coordinate Measuring
Machines (CMM)Techniques for determining the uncertainty of
measurementPart 4: Estimating task-specific uncertainty using
simulation, ISO 15530-4 (draft), Geneva, October 2004.
Now! The Most Accurate Force Calibrations
Ever From A commercial Lab!
0.002% of load through 120,000 lbf
INSTRUMENT CO.
G
A true primary standard
G
Every weight calibrated directly by NIST
G
Accredited to ISO 17025
G
Calibrating load cells, proving rings, force gauges
-- in compression or tension
-- kilograms or Newtons, too
G
Calibrations also available from 0.1 to
1,000,000 lbf
Want your own
dead weight force
machine?
Well build one for you
--from 50 to 120,000 lbf
From the World leader in accurate force calibrations since 1925
For complete details, toll free 1-866-843-0081
1742 Sixth Avenue, York PA 17403-2675 www.mhforce.com Fax 1-717-846-4193
TECHNICAL PAPERS
1. Introduction
Techniques for measuring microwave power have a long history.
In 1967 Rumfelt and Elwell [1] very thoroughly described the
basic theory that underlies microwave power measurements.
Other good overviews include Agilent Technologies application
note on microwave power measurements [2], as well as several
chapters of Engens book on microwave circuit theory. [3] There
are many other documents that detail different aspects of
microwave power measurements. In all of these, however, there
is little to no discussion of the microwave leakage energy that
passes through the microwave power sensors and emerges on
the dc leads of the attached power meter.
This leakage signal is energy that is not accounted for in the
sensing element of the detector and has unintended effects on
the electronics attached to the sensor. The leakage signal occurs
at the lower frequencies used for thermistor type power sensors.
We initially noted and reported this problem in 2006 [4], but at
that time we did not realize the full extent of the problem. We
should first look at the overall measurement process to better
understand the problem.
2. Measurement Process
The main system used for our power measurements is of the
direct comparison type. Its block diagram is shown in Fig. 1.
Power measurements are made by transferring the effective effi-
ciency from a primary power sensor to the splitter/monitor and
then back out to the Device Under Test (DUT). Note that we do
not measure absolute power, but instead measure the effective
efficiency of the power sensors (effective efficiency is the ratio
Power Measurements
1
Ronald A. Ginley, Denis X. LeGolvan and Ann F. Monke
Abstract: Because microwave power measurements are used to support almost every segment of the microwave elec-
tronics industry, the accuracy of these measurements is critical. Recently, several different problems have been found
that affect microwave power measurements, including leakage, compensation bead bias conditions, and common mode
noise. One of the main problems is leakage of microwave energy through the microwave power sensors. This paper
explores some of the effects this leakage energy has on different parts of microwave power measurement systems and
the power measurement process and then discusses possible ways of handling the errors associated with this energy.
Ronald A. Ginley, Denis X. LeGolvan, and Ann F. Monke
Electromagnetics Division
325 Broadway
Boulder, CO 80305 USA
Email: rginley@boulder.nist.gov 1
U.S. Government work, not subject to U. S. copyright.
TECHNICAL PAPERS
of the power read by the electronics of a
sensor to the net power absorbed by the
detector).
The full calibration/measurement
process for the direct comparison type of
system is described by the equation
[2, 5]:
(1)
where:
DUT
is the effective efficiency of the
DUT;
Std
is the known effective efficiency
of the primary transfer standard;
dc-DUT
is the dc-substituted power
measured by the DUTs electronics
when the DUT is measured;
M-Std
is the monitor power when the
system was calibrated;
gl-Std
is the mismatch factor for the
standard (defined below);
dc-Std
is the dc-substituted power
measured by the standards elec-
tronics when the system was cali-
brated;
M-DUT
is the monitor power when
the DUT is measured; and
gl-DUT
is the mismatch factor for
the DUT (defined below).
The mismatch factors are defined by:
(2)
where:
g
is the complex reflection coeffi-
cient looking into the test port of
the splitter;
Std
cient of the primary transfer stan-
dard; and
DUT
cient of the DUT.
The effective efficiency of the DUT
depends directly on the effective effi-
ciency of the standard and the mismatch
factors. It is also directly dependant on
the powers that are determined during
the calibration of the system and the
measurement of the DUT. In the rest of
the paper we will explore how the
leakage energy affects the various factors
outlined above.
3. The Basic Coaxial Thermistor
Detector and Leakage
Figure 2 shows a generic diagram of a
thermistor detector. Initially, with no
microwave signal applied, dc energy is
supplied by a power meter through the
dc connector of the device to the meas-
urement thermistor bead, keeping it at a
constant resistance. As microwave
energy is then applied to the microwave
connector of the device, a dc blocking
capacitor removes the dc energy in the
signal. The microwave energy is then
incident on the measurement thermistor
of the detector. The power meter
attached to the dc connector keeps the
resistance of the thermistor at a constant
resistance by removing enough dc energy
to compensate for the incident
microwave energy. There are filter sec-
tions between the thermistor and the dc
connector to prevent leakage of the
microwave signal into the dc circuit
portion; some leakage occurs because
these filter sections are not sufficient to
stop all of the microwave energy. The
filters are especially ineffective in the
low-frequency range of most coaxial
detectors. The basic structure of rectan-
gular waveguide detectors is much differ-
ent from that of coaxial sensors and the
leakage energy occurs through a much
different mechanism and does not inter-
act in the same way as coaxial detectors.
This paper will deal only with coaxial
detectors.
A fundamental assumption underlying
the use of thermistor sensors is that the
DUT

Std
,
P
dc-DUT
P
M-Std
M
gl-Std
P
dc-Std
P
M-DUT
M
gl-DUT
=
=
1
g
2
1
Std
2
1
2
M
glStd
= M
glDUT
,
Std
1
g
2
1
DUT
2
1
2
DUT
Figure 1. Direct comparison power measurement system.
100 kHz to
20 GHz
Signal
Generator
Resistive
Power
Splitter
Standard
or DUT
Monitor
Power
Sensor
DVM
DVM
Power
Meter
Power
Meter
Figure 2. Generic thermistor detector.
dc
Connector
Microwave
Filter
Measurement
Thermistor
Compensation
Bead
dc
Block
Microwave
Connector
and
TECHNICAL PAPERS
thermistor interacts with all of the
microwave energy that is incident on it.
The leakage energy violates this assump-
tion. The leakage energy basically passes
through the detector without ever affect-
ing the thermistor. This lack of interac-
tion represents a direct error in the
measurement of effective efficiency. We
will now examine the basic setup
employed to investigate the leakage
effect.
4. Experimental Setup
Figure 3 shows the equipment used to
test the effects of leakage. The filter sec-
tions indicated in the figure are low-pass
filters with a cutoff frequency of approx-
imately 50 kHz. The filter attenuates
signals at and above 100 kHz by at least
40 dB. The design of the filter is shown
in Fig. 4. Figure 3a shows the vector
network analyzer (VNA), detector and
power meter with the low pass filter.
Figure 3b shows the setup of the direct
comparison system with filters and
power meters to bias the compensation
beads of the detectors if they had such
beads (compensation thermistors are
used to compensate meter readings for
temperature drift). Not all coaxial ther-
mistor detectors have compensation
beads. The ones that do not have them
still exhibit strong leakage signals.
Experiments were carried out with both
types of detectors. The purpose of the
ferrite cores on the digital voltmeter
(DVM) leads is to limit common mode
noise (common mode noise is the noise
signal present equally on two conductors
with respect to some reference point).
5. Overall Difference in Response
Due to System Changes
The overall effect of adding filters,
biasing compensation beads, and use of
DVM lead ferrite cores on reflection
coefficients and thus effective efficiency
is shown in Fig. 5. This figure shows the
results from the measurement system
when no filters, etc. are used for any part
of the measurement and the results
when the filters, etc. are used for all parts
of the system. This includes filters
between the monitor, standard, DUT and
compensation power meters, and reflec-
tion coefficients (
s
) measured with the
filters in place and compensation beads
biased. Additionally in Fig. 5, the values
for the effective efficiency of the device
as determined by the voltage/impedance
(V/I) technique are shown. Note that the
effective efficiency with no filter exceeds
1.0, which is not physically possible for
the particular type of detector used
(energy would be generated from
nothing; a good idea, just not possible).
This clearly indicates that there is some-
thing happening in the system that is not
following the basic assumptions of the
analysis. The inset to Fig. 5 shows the
difference between efficiency results
with and without filters, etc. It is seen
that there is a significant difference
below approximately 600 kHz.
6. Interactions with the Primary
Transfer Standard
As can be seen from equation (1),
DUT
is directly linked to
Std
. The important
consideration here is that when the stan-
dard is used to calibrate the system, it
must be used under the same conditions
as when it was originally calibrated. Thus
if the standard was originally calibrated
with the compensation beads biased and
filters used to block the leakage from
entering the power meter, then it needs
to be used with the compensation beads
biased and filters in front of the power
meter when used to calibrate the direct
comparison system. This is also true
when determining the reflection coeffi-
cient of the standard.
Figure 3. Experimental setup: (a) for measuring reflection coefficient; (b) for effective effi-
ciency measurement on the direct comparison system.
DVM
Filter
Filter
Filter Filter
Detector
Detector
Detector
DVM
Power
Meter
Power
Meter
(a) (b)
Power
Splitter
VNA
Power
Meter
Power
Meter
Figure 4. Design of low pass filter used in measurements.
220 H 330 H 220 H
0.05 F
0.1 F 0.1 F
0.05 F
TECHNICAL PAPERS
The low frequency standards used at
NIST (100 kHz to 80 MHz) were cali-
brated using the voltage and impedance
technique [4]. The power meter used for
the calibration biased the compensation
beads of the standard, and a filter was
used. To show the effect of different stan-
dard conditions, the direct comparison
system was first calibrated with no filter
between the standard and the power
meter, and the compensation bead of the
standard was not biased. A device was
then measured based on this calibration
(the measured device did use the filter
and biased comp beads). Next, the filter
was inserted between the standard and
the power meter, the standard compensa-
tion beads were biased, and the system
was again calibrated. After that, the
same device was measured with the same
filter/comp bead conditions. There was a
significant change in the measurements
for these two different conditions, as
seen in Fig. 6. Shown in the figure is the
difference between the measurement
taken without filters, etc. and the meas-
urement taken with filters, etc. There is a
difference at 100 kHz (where the leakage
should be the greatest) of greater than
0.01 (absolute value), which is over
twice the uncertainty of the measure-
ments.
7.
DUT
and DUT Reflection
Coefficient Changes
It would first appear that there should be
no change in of the DUT with leakage.
The power meter is holding the thermis-
tor at a constant resistance, and this
resistance (50 for the microwave
signal) is the basis for . We have found,
however, that there is an effect. Figure 7
shows the difference between the meas-
urement of with and without a filter.
The effect is very strong at 100 kHz
where the leakage is greatest and rapidly
disappears as the frequency increases.
To examine the significance of small
changes in , in Fig. 8 we show how
DUT
varies with changes in
DUT
magni-
tude from 0 to 0.2 and phase from 0 to
180. To arrive at the plots in the figure,
equation (1) was used with the following
quantities held fixed:
Std
= 0.95
dc-DUT
= 7.0 mW
M-Std
= 7.1 mW
dc-Std
= 7.1 mW
M-DUT
= 7.0 mW
g
= 0.01 at 30
Std
= 0.01 at 60
From the figure, it is seen that when
the magnitude of
DUT
is small, there is
a much smaller change in
DUT
than
when
DUT
is larger. Also, when
DUT
is
larger, the phase variation has a much
larger effect. At 100 kHz, for a typical
detector,
DUT
is at its largest (approxi-
mately 0.2), thus
DUT
will be much
more sensitive to changes in
DUT
.
8. Compensation Beads
It was originally believed that the effects
of biasing the compensation beads, for
those detectors that have them, appeared
only at high frequencies (above ~15
GHz). [4] There are also small changes
Figure 5. Comparison of measurements with (pink) or without (blue) filters, compensation
beads, etc. Inset shows the difference between the two sets of results.
0 2 4 6 8 10
E
f
f
e
c
t
i
v
e
E
f
f
i
c
i
e
n
c
y
Frequency (MHz)
1.005
1.000
0.995
0.990
0.985
0.980
Without Filter
With Filter
Figure 6. Difference between a measurement based on a calibration with no filters, etc and
a measurement based on a calibration with filters, etc.
0 2 4 6 8 10
D
i
f
f
(
N
o
F
i
l
t
e
r
s
+
C
a
l
F
i
l
t
e
r
s
+
C
a
l
)
0.004
0.002
0.000
-0.002
-0.004
-0.006
-0.008
-0.010
-0.012
Frequency (MHz)
TECHNICAL PAPERS
in effective efficiency that have been
observed at low frequencies (below 100
MHz) due to the bias condition of the
compensation beads. While the changes
are small, on the order of 0.0005, they
cannot be neglected. The underlying
reason for the compensation bead effect
is complicated, but in general it is most
likely due to a more complex leakage
interaction.
9. Higher Frequency Detectors
While this paper has focused on the low
frequency microwave power detectors,
the leakage problem exists with higher
frequency detectors as well. Manufactur-
ers try to get the most out of their
detectors by using them over the largest
frequency range possible. The
microwave energy cannot be adequately
contained over these ranges. Leakage
will occur at the low frequencies, where
filtering the signal is difficult. Initial
studies show that there is a large leakage
problem for the class of thermistors that
are rated for 10 MHz to 18 GHz. More
investigation is needed for these devices.
10. Summary
This paper investigated the effects of
microwave signal leakage through
microwave power sensors. It was shown
that there is a significant change in the
results of a measurement when this
occurs. By utilizing low-pass filters
between the detectors and their associ-
ated power meters, the leakage effect can
be minimized. Special care must be
taken to use the measured devices under
the same conditions as when they were
originally measured.
11. References
[1] A.Y. Rumfelt and L.B. Elwell, Radio
frequency power measurements, Proc.
IEEE, vol. 55, no. 6, pp. 837850, June
1967.
[2] Hewlett-Packard, Fundamentals of rf
and microwave power measurements,
Hewlett-Packard Application Note 64-1,
August 1977. [Note: This Application
Note has been updated by Agilent Tech-
nologies in 2001 as Application Note 64-
1C which is available at the web site
www.agilent.com.]
[3] G.F. Engen, Microwave Circuit Theory
And Foundations of Microwave Metrol-
ogy, Peter Peregrinus Ltd., London, UK,
1992.
[4] R.A. Ginley, A Direct Comparison
System for Measuring Radio Frequency
Power (100 kHz to 18 GHz), Proceed-
ings of the 2006 Measurement Science
Conference, 2006.
[5] J.R. Juroshek, NIST 0.05-50 GHz
Direct-Comparison Power Calibration
System, Proceedings IEEE CPEM 2000,
May 2000.
Figure 7. Difference between measurements of the reflection coefficient of a detector with
and without the filter.
D
i
f
f
(
G
a
m
m
a
w
i
t
h
w
i
t
h
o
u
t
f
i
l
t
e
r
)
Frequency (MHz)
0.016
0.014
0.012
0.010
0.008
0.006
0.004
0.002
0.000
-0.002
Figure 8. Sensitivity of
DUT
to changes in
DUT
. Changes in magnitude of
DUT
are
shown along the bottom axis; the different plots show changes in the phase of
DUT
.
0.00 0.05 0.10 0.15 0.20
E
f
f
e
c
t
i
v
e
E
f
f
i
c
i
e
n
c
y
(
D
U
T
)
Magnitude of DUT Reflection Coeffcient (
DUT
)
1.000
0.995
0.990
0.985
0.980
0.975
0.970
0.965
0.960
0.955
0.950
0.945
0 Deg.
45 Deg.
90 Deg.
135 Deg.
180 Deg.
0 2 4 6 8 10
TECHNICAL PAPERS
1. Introduction and Objective
Within the framework of the Inter-Amer-
ican Metrology System (SIM), no previ-
ous comparison activity had been carried
out between the participants of the
present comparison in mass. The objec-
tive of this intercomparison is to
compare mass measurements at 200 mg,
1 g, 50 g and 200 g and to estimate the
degrees of equivalence and levels of
measurement agreement between the
two laboratories.
The nominal values of the weights are
the same as some of those used for
SIM.M.M-S2 (see Appendix B of Refer-
ence [1]) comparison, in order to link
the results of this comparison with the
Bureau International des Poids et
Mesures (BIPM) Consultative Commit-
tee for Mass and Related Quantities
(CCM) key comparisons CCM.M-K4 and
CCM.M-K5 (see Appendix B of Refer-
ence [1]).
At the time of the publication of this
paper, Venezuela has not declared Cali-
bration and Measurement Capabilities
(CMCs) within the International Com-
mittee for Weights and Measures (CIPM)
Mutual Recognition Arrangement
(MRA) framework, and this comparison
will be useful in that process. At the time
of the comparison, SENCAMER was
offering calibration services of OIML F
1
[2] weights from 1 mg to 5 kg. CMCs
declared by Chile can be found in the
BIPMKey Comparison Data Base (BIPM
KCDB), Appendix C of Reference [1].
2. Comparison Process
2.1 General Guidelines
A measurement protocol was previously
agreed and reported to the SIM Mass &
Related Quantities Metrology Working
Group. It stated the following relevant
technical aspects:
Measurements were done after the
acclimatization time as specified in
OIML R111 for class E
1
. [2]
The participating laboratories meas-
ured the conventional mass of the arti-
facts according to OIML R 111. [2]
No washing was performed. Before
measurements, dust particles were
removed from the surface of the stan-
dard by a soft brush.
All weightings were performed in air.
Uncertainties were estimated and com-
A Comparison in Conventional
Mass Measurements Between
SENCAMER (Venezuela) and
CESMEC-LCPN-M (Chile)
Francisco Garca, Elas Salazar, Ral Hernndez, Luis Rojas, Haygas Kalustian and Fernando Leyton
Abstract: A conventional mass comparison was carried out between the Servicio Autnomo Nacional de Normal-
izacin, Calidad, Metrologa y Reglamentos Tcnicos (SENCAMER, Venezuela) and Laboratorio Custodio de los
Patrones Nacionales de Masa at CESMEC Ltda. (CESMEC-LCPN-M, Chile), in order to estimate the degrees of equiv-
alence for calibration of a mass artifact and the uncertainty associated with its measurement. This comparison was
carried out using the following nominal values: 200 mg, 1 g, 50 g and 200 g. The results obtained by each laboratory
are presented in this document. Evaluations of correlations among measurement results are considered because
CESMEC-LCPN-Mhad previously calibrated SENCAMERs reference standards. For all mass artifacts, the normalized
error, E, ranged from 0.05 to 0.91, indicating that SENCAMER has calibration capabilities for OIML F
1
weights. In
addition, correlations were found to be not significant in the evaluation of the degrees of equivalence and levels of meas-
urement agreement.
Francisco Garca
Ral Hernndez
Fernando Leyton
CESMEC-LCPN-Mass
Av. Marathon 2595
781-0552, Macul, Chile
Email: fgarcia@cesmec.cl
Elas Salazar
Luis Rojas
Haygas Kalustian*
SENCAMER
Zona Industrial San Vincente II
Calle H, Maracay, Venezuela
*Currently working at Ministerio de
Ciencia y Tecnologa, Caracas, Venezuela
TECHNICAL PAPERS
bined according to the ISO Guide to the Expression of
Uncertainty in Measurement. [3]
The standards were transported by hand in order to assure
the conventional mass stability of the test objects.
2.2 Comparison Objects
The weight standards, or test objects, were provided by
CESMEC-LCPN-M and have the nominal values and densities
as listed in Table 1.
Density values were determined according to method F
1
in
Reference [2]. The weights were manufactured by Hfner
Gewichte GMBH
3
, Hohenhardtsweiler Strasse 4, DE 74420
Oberrot, Germany.
2.3 Comparison Round
The comparison was performed in round including an initial and
a final measurement at the pilot laboratory, which was CESMEC-
LCPN-M. See Table 2 for a listing of the standards used by each
laboratory, the date of measurement for the comparison, and the
institute that performed the calibration of the standard.
3. Results
Results as reported by each laboratory for each nominal value
and the expanded uncertainty (k 2) are presented in Table 3.
Also listed is the calibration certificate identification issued for
the comparison measurement.
4. Discussion
4.1 Stability of the Test Objects
For the evaluation of the stability of the test objects during the
course of the measurements, let us define the following quantities:
m
ct
(i)
is the initial measurement result by CESMEC-LCPN-M.
m
ct
(f)
is the final measurement result by CESMEC-LCPN-M.
m
cr
is the conventional mass value of CESMEC-LCPN-Ms
reference standard.
r
is the density of CESMEC-LCPN-Ms reference standard.
t
is the density of the test object.
Table 1. Density of the comparison objects.
Nominal Value Density
Expanded uncertainty
of the density value
(k = 2)
200 g
8009 kg/m
3
47 kg/m
3
50 g
8009 kg/m
3
47 kg/m
3
1 g
8009 kg/m
3
47 kg/m
3
200 mg
8600 kg/m
3
57 kg/m
3
Table 2. Participant laboratories and standards.
Laboratory Country
Standard used for
the comparison
Institute that calibrated
the standard
Date of measurement
for the comparison
SENCAMER Venezuela Set G952734 CESMEC-LCPN-M 2005/04/28
CESMEC-LCPN-M
(Pilot)
Chile Set 122141/98
Landesamt fr Mess-und
Eichwesen Brandenburg
(DKD-K-0601)
2005/04/18 and
2005/05/16
3
Certain commercial equipment, instruments, or materials are identi-
fied in this paper in order to adequately describe the experimental
procedure. Such identification does not imply recommendation or
endorsement by the author or NCSL International, nor does it imply
that the materials or equipment identified are the only or best avail-
able for the purpose.
Laboratory Conventional mass value
Uncertainty
(k = 2)
Calibration Certificate
Issued for the measurement
CESMEC-LCPN-M
(Initial)
200 mg +0.000 mg
1 g +0.012 mg
50 g +0.04 mg
200 g +0.14 mg
0.006 mg
0.010 mg
0.03 mg
0.10 mg
396/DKD-K-28401/05-04
SENCAMER
200 mg 0.020 mg
1 g +0.01 mg
50 g +0.04 mg
200 g 0.04 mg
0.020 mg
0.03 mg
0.10 mg
0.30 mg
C/LM/SV/04/05/0228
CESMEC-LCPN-M
(Final)
200 mg 0.001 mg
1 g +0.015 mg
50 g +0.03 mg
200 g +0.14 mg
0.006 mg
0.010 mg
0.03 mg
0.10 mg
413/DKD-K-28401/05-05
Table 3. Results as reported by each participant.
TECHNICAL PAPERS
a
(i)
is the air density during the initial measurement of the
comparison object at CESMEC-LCPN-M.
a
(f)
is the air density during the final measurement of the
comparison object at CESMEC-LCPN-M.
m
(i)
is the initial conventional mass difference between the
test weight and CESMEC-LCPN-Ms reference standard.
m
(f)
is the final conventional mass difference between the
test weight and CESMEC-LCPN-Ms reference standard.
m
ct
(i)
and m
ct
(f)
where calculated according to Reference [2]. It is
useful to write their formulas for the discussion that follows:
(1)
(2)
The stability of the test object is evaluated by the difference, ,
between equations (1) and (2) applying the normalized error, E,
and criteria stated in Reference [4]:
(3)
where U() is evaluated applying formulas (13) and (18) in Ref-
erence [3] with k 2 and is given by:
(4)
Reference [4] assumes independence of results but for consis-
tency with Reference [3] the correlation term and u(m
ct
(f)
, m
ct
(i)
)
is not neglected and is evaluated using equation (F.2) in Refer-
ence [3], which is the following:
(5)
Disregarding second order terms and considering that
a
(i)

a
(f)

a
1.12 kg/m
3
, we have:
(6)
E values obtained from equation (6) are presented in Table 4.
Since there is good agreement between results, | E| 1 for all
nominal values. Thus, there is no evidence of a significant drift
effect over the period of the measurements, and it will not be
considered as an uncertainty component when evaluating the
degrees of equivalence and measurement level agreements
between both laboratories.
Table 5 shows alternative evaluations of equation (4). In the
second column variances and covariances are considered while
in the third column covariances are not included. The differences
are all less than 8 %which is negligibly small.
4.2 Degrees of Equivalence and Level of Measurement
Agreement
The pair (d,U(d)) is called degrees of equivalence [1] and is cal-
culated for each nominal value. d is obtained by the difference
between the measurement result by SENCAMER, m
ct
(s)
, and the
mean of the results by CESMEC-LCPN-M:
(7)
The measurement result from SENCAMER, m
ct
(s)
, is given by:
(8)
where,
m
cr
(s)
is the conventional mass value of SENCAMERs refer-
ence standard.
r
(s)
is the density of SENCAMERs reference standard.
m
ct
(s)
m
cr
,
(s)
j
(
,
1+(
1
j
(
,
\
,
(
\
,
(
+ m
a
(s)
o
)
t r
(s)
(s)
E ,
U( )
m
ct
(f)
m
cr
1+
( )
1
j
(
,
\
,
(
j
(
,
\
,
(
+ m .
a
(f)
o
t r
(f)
m
ct
(i)
m
cr
1+
( )
1
j
(
,
\
,
(
j
(
,
\
,
(
+ m
a
(i)
o
t r
(i)
U( ) = 2 u
2
(m
ct
) + u
2
(m
ct
) +
(f) (i)
]
]
]
]
2 u(m
ct ,
m
ct
) .
(f) (i)

m
ct
m
ct
(f) (i)

]
]
]
]
(f) (i)
u(m
ct ,
m
ct
)
m
ct
m
ct
(f) (i)
m
cr
m
cr
u
2
(m
cr
)+
m
ct
m
ct
(f) (i)
t
u
2
(
t
)+

m
ct
m
ct
(f) (i)
r
u
2
(
r
) .

d
(s)
m
ct
.
m
ct
+ m
ct
\
,
(
j
(
,
2
(i) (f)
2 u
2
(m
ct
)+u
2
(m
ct
)2u
2
(m
cr
)
(f) (i)
]
]
]
]
2

2m
cr
( a o)
2
+
.
]
]
]
]

u
2
(
t
) u
2
(
r
)
t
4
r
4
\
,
(
\
,
(
E =
_________________________
m
ct
= m
ct
(f) (i)
_________________________
Table 4. Measurement agreement between initial and final results
by the pilot laboratory.
Nominal Value E
200 mg
1 g
50 g
200 g
0.13
0.22
0.25
0.00
Table 5. Evaluation of U() with and without the contribution of
covariances.
Nominal Value
Variances and
covariances
Variances only
U( )
from equation (4)
2
.
[u
2
(m
ct
(f)
)+u
2
(m
ct
(i)
)]
200mg 0.008mg 0.008mg

1 g 0.013mg 0.014mg
50 g 0.04mg 0.04mg
200g 0.13mg 0.14mg
TECHNICAL PAPERS
a
(s)
is the air density during the measurement of the com-
parison object at SENCAMER.
m
(s)
is the conventional mass difference between the test
object and SENCAMERs reference standard.
There are correlations among m
ct
(s)
, m
ct
(i)
and m
ct
(f)
because
CESMEC-LCPN-M used the same reference standards (Set
122141/98) to calibrate both SENCAMERs reference stan-
dards (see Table 3) and the comparison objects. Then U(d) is
expressed as:
(9)
In order to quantify the covariances in the evaluation of U(d),
m
ct
(s)
is re-written in terms of using the weighing equation that
was used to calibrated the mass standards of SENCAMER:
(10)
where:
a
is the air density during the calibration of SEN-
CAMERs reference standard at CESMEC-LCPN-M
m
is the conventional mass difference between the SEN-

CAMERs reference standard and CESMEC-LCPN-Ms
reference standard.
From equations (8) and (10), m
ct
(s)
becomes:
(11)
Then, m
cr
,
t
and
r
can be identified as common independent
variables in the set of equations (1), (2) and (11) and the covari-
ances can be evaluated using equation (F.2) in Reference [3]:
(12)
(13)
(14)
After applying the approximations
a
(s)

a
(i)

a
(f)

a
(so both
laboratories are almost at the same height above sea level 500 m,
a
(s)
1.14 kg/m
3
and
a
(i)

a
(f)
1.12 kg/m
3
) and m
cr
(s)
m
cr
,
U(d) can be expressed as:
(15)
The degrees of equivalence are presented in Table 6, where it
can be seen that for all masses the nominal values | E| 1.
On the other hand, taking on account that the expanded
uncertainties reported by SENCAMER, which are 3 times
smaller than the maximum permissible errors for class F
1
and
the agreement of the results, it can be concluded that SEN-
CAMER has good calibration capabilities for F
1
weights in the
nominal values considered within the framework of this com-
parison.
Finally, the contributions of the covariances in the evaluation
of U(d) were not relevant, so the difference between the evalu-
ations with and without them are equal, as shown in Table 7.
The total contribution of variances is larger than covariances
and there is a dominant variance which is related to m
ct
(s)
deter-
mination (U(d) U(m
ct
(s)
)). This is a positive fact so the other
uncertainty contributions do not produce a screening effect in
the evaluation of SENCAMER measurement capabilities,
degrees of equivalence and levels of measurement agreement.
5. Conclusions
There are satisfactory levels of measurement agreement between
measurement results for 200 mg, 1 g, 50 g, and 200 g between
SENCAMER and CESMEC-LCPN-M. The normalized error
ranged from 0.05 to 0.91. Degrees of equivalence are listed in
Table 5.
In the nominal values of this comparison, SENCAMER has
calibration capabilities for OIML F
1
weights.
m
cr
m
cr
,
(s)
j
(
,
1+(
1
j
(
,
\
,
(
\
,
(
+ m
a
o
)
r r
(s)
U(d)=
u
2
(m
ct
)+u
2
(m
ct
)+u
2
(m
ct
)+2 u(m
ct ,
m
ct
)+
.
(s) (i) (s) (i)
dd
m
ct
m
ct
(s) (i)
]
]
]
]
]
]
]
]
]
]
]
]
(f)
+2 u(m
ct ,
m
ct
)+2 u(m
ct ,
m
ct
)
(s) (i) (f)
dd
m
ct
m
ct
(s) (f)
(f)
dd
m
ct
m
ct
(i) (f)
2
+m .
j
(
,
1+(
1
j
(
,
\
,
(
\
,
( a
(s)
o
)
t r
(s)
(s)
m
ct
m
cr
(s)
j
(
,
1+(
1
j
(
,
\
,
(
\
,
(
+ m
a
o
)
r r
(s)
j
(
,
\
,
(
U(d)2 u
2
(m
ct
)+u
2
(m
ct
)+u
2
(m
ct
)u
2
(m
cr
)
(s)
(i)
]
]
]
]
1
4
1
4
(f) 3
2
2

m
cr
(

a

o)
2

+
]
]
]
]

u
2
( t) u
2
( r)

t

r
\
,
(
\
,
(
3
2
(i) (f)
u(m
ct ,
m
ct
)
m
ct
m
ct
(i) (f)
m
cr
m
cr
u
2
(m
cr
)+
m
ct
m
ct
(i) (f)
t
u
2
(
t
)+

m
ct
m
ct
(i) (f)
r
u
2
(
r
),

(s) (f)
u(m
ct ,
m
ct
)
m
ct
m
ct
(s) (f)
m
cr
m
cr
u
2
(m
cr
)+
m
ct
m
ct
(s) (f)
t
u
2
(
t
)+

m
ct
m
ct
(s) (f)
r
u
2
(
r
)

(s) (i)
u(m
ct ,
m
ct
)
m
ct
m
ct
(s) (i)
m
cr
m
cr
u
2
(m
cr
)+
m
ct
m
ct
(s) (i)
t
u
2
(
t
)+

m
ct
m
ct
(s) (i)
r
u
2
(
r
)

Table 6. Degrees of equivalence and levels of measurement

agreement.
Nominal Value
d
(mg)
U(d)
(mg)
E
200 mg 0.020 0.020 0.91
1 g 0.004 0.031 0.13
50 g 0.01 0.10 0.05
200 g 0.18 0.32 0.59
TECHNICAL PAPERS
Even with correlations among results,
their values were found to be not signif-
icant in the evaluation of the degrees of
equivalence and levels of measurement
agreement.
6. Acknowledgements
This comparison was performed within
the framework of the Physikalisch-Tech-
nische Bundesanstalt cooperation project
in Venezuela: Promotion of the national
metrology service MSTQ (Project No.:
1995.2032.1 / PTB No.: 95010).
7. References
[1] Mutual Recognition of National Meas-
urement Standards and of Calibration
and Measurement Certificates Issued by
National Metrology Institutes, Mutual
Recognition Arrangement, International
Committee for Weights and Measures
(MRA-CIPM), Paris, 14 October 1999.
(Available at: http://www.bipm.org/en/
cipm-mra/mra_online.html)
[2] Weights of classes E
1
, E
2
, F
1
, F
2
, M
1
,
M
12
, M
2
, M
23
and M
3
. Part 1: Metro-
logical and Technical Requirements,
OIML R 111-1, International Organiza-
tion Of Legal Metrology (OIML), 2004.
(Available at http://www.oiml.org/pub-
lications/R/R111-1-e04.pdf)
[3] Guide To The Expression Of Uncer-
tainty In Measurement, International
Organization for Standardization, ISO
TAG 4 WG 3, BIPM, IEC, IFCC, ISO,
IUPAC, IUPAP, OIML; Geneva, Switzer-
land, 1995.
[4] ISO Guide 43-1, Proficiency Testing by
Interlaboratory Testing, Part 1: Develop-
ment and Operation of Proficiency
Testing Schemes, International Organ-
ization for Standardization, Geneva,
Switzerland, 1996.
Table 7. Evaluation of U(d) with and without the contribution of covariances.
Nominal Value
Variances and
covariances
Variances only
U(d)
from equation (15)
2
.
[u
2
(m
ct
(s)
)+
4
1
u
2
(m
ct
(i)
)+
4
1
u
2
(m
ct
(f)
)]
200 mg 0.020 mg 0.020 mg

1 g 0.031 mg 0.031 mg
50 g 0.10 mg 0.10 mg
200 g 0.31 mg 0.31 mg
Humidity goes Digital

Features
Leading the class: the new generation of humidity generators from Rotronic
HygroGen distinguishes itself in the marketplace
for its precise generation of controlled humidity
and temperature environments. High chamber
stability and low gradients result in an unmatched
price / performance ratio.
To find out more, contact us today.
Generates a precise humidity/temperature
environment
Minimal chamber temperature gradients
Excellent chamber stability
Quick to set point
Wide operating range (595% RH, 550C)
Completely self contained
160 E. Main St Huntington, NY 11743 Tel. 631-427-3898 Fax. 631-427-3902 www.rotronic-usa.com
REVIEW PAPERS
1. Introduction
In recent years, the American Society of Mechanical Engineers
(ASME) B89 Committee on Dimensional Metrology has pub-
lished a number of American National Standards and Technical
Reports. The scopes of these documents cover a variety of topics,
including the evaluation of measuring devices and systems,
measurement uncertainty, decision rules, risk analysis and trace-
ability. The published documents also include revisions to previ-
ously published standards, such as plug and ring gages, that
reflect technological advances and the perceived need to
provide a loosely defined measurement uncertainty statement.
Overview of ASME B89 Standards
with an Emphasis on B89.4.22
Methods for Performance Evaluation
of Articulated Arm Coordinate
Measuring Machines
1
Brian Parry
Abstract: This paper is an overview of the new and recently reaffirmed standards developed by the American Society
of Mechanical Engineers (ASME) B89 Committee on Dimensional Metrology. Detailed information is provided for
ASME B89.4.22-2004Methods for Performance Evaluation of Articulated Arm Coordinate Measuring Machines,
which was written jointly by users, manufacturers and representatives from academia and the National Institute of Stan-
dards and Technology (NIST). The B89.4.22 standard addresses the performance evaluation of articulated arm coor-
dinate measuring machines (AACMMs) by specifying a minimum set of requirements that can be used to perform a
reasonable evaluation of machine performance. Using standardized definitions and test procedures, users can deter-
mine if an AACMM is appropriate for their requirements. The AACMM standard is also intended to provide a com-
parative basis to evaluate machines from different suppliers. Finally, since a more complex evaluation may be
appropriate for some applications, special methods can be detailed and specified using the AACMM Specification
Sheets.
Brian Parry, P.E.
Chair, ASME B89 Dimensional Metrology Committee
The Boeing Company
P. O. Box 3707; Mail Code 14-27
Seattle, WA 98124-2207 USA
Email: brian.parry@boeing.com
1
This paper won a Best Paper Award at the 2007 NCSLI Workshop
and Symposium in the category of Management and Quality.
REVIEW PAPERS
This paper provides an overview of these new and reaffirmed
standards, with a particular emphasis on ASME B89.4.22-2004
Methods for Performance Evaluation of Articulated Arm Coor-
dinate Measuring Machines [1, 2], written jointly by users, man-
ufacturers and representatives from academia and the National
Institute of Standards and Technology (NIST).
Due to the ambiguity of manufacturers specifications for
commercial AACMMs, comparing their performance has been
extremely difficult, if not impossible. Because of this and the
increasing use of this class of measurement equipment, the
ASME B89 Standards Committee accepted the task of writing
an industry standard applicable to these machines. Where pos-
sible, the AACMM standard parallels ASME B89.4.1b-2001 for
conventional coordinate measuring machines. Part of the
impetus to write the AACMM standard was based on an evalu-
ation of the applicability of the B89.4.1 standard [2] (then
B89.1.12) conducted by NIST.
The AACMM standard addresses the performance evaluation
of AACMMs by providing standardized definitions and test pro-
cedures. These procedures enable users to determine if an
AACMM is appropriate for their specific requirements, and to
compare and evaluate machines from different suppliers. Con-
siderable time and energy was devoted to documenting the eval-
uation and presenting the data analysis. As with all ASME B89
documents, the intent of the AACMM standard is to specify a
minimum set of requirements that can be used to obtain a rea-
sonable evaluation of performance. The Committee recognized
that a more complex evaluation may be appropriate for some
applications. To facilitate the supplier and user communication,
any of these special methods can be detailed and specified
using the AACMM Specification Sheets.
2. Brief History of ASME
The American Society of Mechanical Engineers (ASME) was
founded on 16 February 1880 by a group of prominent engi-
neers, led by Alexander Holley, Henry Worthington and John
Sweet, in New York City. The Institution of Chartered Mechan-
ical Engineers had been successfully established in Great Britain
in 1847. In the United States, the American Society of Civil
Engineers had been active since 1852 and the American Insti-
tute of Mining Engineers was organized in 1871. At the formal
inaugural meeting, the business model and organizational struc-
ture of these institutions was adopted by ASME.
Today, ASME is a worldwide engineering society focused on
technical, educational and research issues. It has over 125,000
members and conducts one of the worlds largest technical pub-
lishing operations, holds some 30 technical conferences and 200
professional development courses each year, and sets many
industrial and manufacturing standards.
2.1 Brief History of the ASME B89 Committee
The ASME Board of Standards formed the B89 Metrology Com-
mittee on 19 October 1963 and Mr. Fullmer was appointed the
first Chairman. The initial Committee structure consisted of six
working groups. Each working group was tasked with develop-
ing a needed standard for a field of metrology, including length,
angles, measurement environment, and so on. The working
groups later evolved into the divisions of B89, and each division
manages a number of project teams.
2.2 B89 Charter
The current B89 Committee Charter is as follows:
Calibration and the specific conditions relating
thereto. It shall encompass the inspection and the
means of measuring the characteristics of the various
geometrical configurations such as lengths, plane sur-
faces, angles, circles, cylinders, cones, and spheres.
Today, the B89 Committee is comprised of seven divisions.
These represent the six original divisions plus Division 7, cov-
ering Measurement Uncertainty and related issues, which was
formed on 22 January 1998.
ASME B89.1 Length
ASME B89.2 Angles *
ASME B89.3 Geometry
ASME B89.4 Coordinate Measuring Technology
ASME B89.5 General Principles and Definitions *
ASME B89.6 Environment *
ASME B89.7 Measurement Uncertainty
Note: Those Divisions marked with an asterisk (*) are
presently dormant.
Note that some groups are now dormant. For example, after a
considerable amount of work, it appeared that industry had no
great need for a new standard on measuring angles. The B89.5
is a compendium of existing, published ASME definitions and
as such is more of a Standard Writers Standard, rather than an
actual standard or technical report. The B89.6.2 [3] document
has essentially been rewritten and published as ISO 16015,
Geometrical Product Specifications (GPS)Systematic Errors
and Contributions to Measurement Uncertainty of Length
Measurement due to Thermal Influences. [4]
The six Technical Reports published by the B89.7 Division
(Measurement Uncertainty) are:
ASME B89.7.2 Dimensional Measurement Planning
ASME B89.7.3.1 Guidelines for Decision Rules: Consider-
ing Measurement Uncertainty in Deter-
mining Conformance to Specifications
ASME B89.7.3.2 Guidelines for the Evaluation of
Dimensional Measurement Uncer-
tainty
ASME B89.7.3.3 Guidelines for Assessing the Reliability
of Dimensional Measurement Uncer-
tainty Statements
ASME B89.7.4.1 Measurement Uncertainty and Confor-
mance Testing: Risk Analysis
ASME B89.7.5 Metrological Traceability of Dimen-
sional Measurements to the SI Unit of
Length
2.3 Brief History of the ASME B89.4.22 Project Team
The initial work on the performance evaluation of AACMMs
was performed by Caskey, et al. and the results published in
NISTIR 5562(R)Application of the ANSI/ASME B89.1.12M
REVIEW PAPERS
Standard to an Articulating Arm Coordinate Measuring
Machine1994. [5] The approach used was to see if the exist-
ing standard for linear axes coordinate measuring machines
(CMMs) could be used for AACMMs . The summary of the
report states, this report demonstrates that applying the B89
Standard is a good starting point for further refinements in
developing an optimized performance test procedure.
As an outcome of the above report, a new ASME Project
Team (PT) was formed and met for the first time on 6 October
1994 at the Fall Meeting in Bellevue, Washington. The PT pro-
ceeded to work on developing a Standard, with active participa-
tion from manufacturers, users, NIST and academia, most
notably the University of North Carolina at Charlotte (UNCC).
3. Review of Recently Published Standards and
Technical Reports
The American National Standards Institute (ANSI) requires
standards to be reviewed every five years and then reaffirmed,
revised or withdrawn. The recently reaffirmed five B89 stan-
dards are primarily for small hand held gages or devices as
shown below, with the exception being the standard for CMM
software evaluation.
ASME B89.1.9 Gage Blocks
ASME B89.1.10 Dial Indicators (for Linear Measure-
ments)
ASME B89.1.13 Micrometers
ASME B89.1.17 Measurement of Thread Measuring
Wires
ASME B89.4.10 Methods for Performance Evaluation of
Coordinate Measuring Systems Soft-
ware
Recently published (or soon to be published) documents
include the following.
ASME B89.4.10360-2 Methods for Performance Evalua-
tion of Coordinate Measuring
Machines
ASME B89.4.19 Performance Evaluation of Laser-
Based Spherical Coordinate Meas-
urement Systems
ASME B89.4.22 Methods for Performance Evalua-
tion of Articulated Arm Coordinate
Measuring Machines
ASME B89.7.Series Measurement Uncertainty and
related issues
ASME B89.4.10360-2 is the latest attempt to harmonize
CMM performance evaluation with its ISO counterpart. ASME
B89.4.19 is a standard to evaluate tracking interferometers,
while ASME B89.4.22 addresses AACMMs. There are no equiv-
alent ISO documents currently available for ASME B89.4.19
and ASME B89.4.22, but both of these standards have been pro-
posed and accepted as new work items in ISO TC 213.
4. Detailed Review of ASME B89.4.22
The ASME B89.4.22 standard [1] consists of six sections, cov-
ering 18 pages, with 16 figures and four tables. In addition there
are 10 appendices covering an additional 26 pages. As is
common in most ASME Standards, the non-mandatory appen-
dices contain what is frequently referred to as the educational
content. A summary of the 10 appendices is shown below.
Appendix A Users Guide to ASME B89.4.22
Appendix B Thermal Environment Testing
Appendix C Seismic Vibration Verification Tests
Appendix D Electrical Power Monitoring Tests
Appendix E Interim Testing of AACMM Systems
Appendix F Ball Bar Design and Mounting Recom-
mendations
Appendix G Kinematic, Conical Seat and Cham-
fered Hole Design Recommendations
Appendix H Determination of Thermal Error Index
Appendix I Statistics Used in Specifying AACMM
Performance Evaluation
Appendix J Application of Decision Rules
4.1 Section 1: Scope
The standard is currently limited to machines with up to seven
rotational joints. Motorized axes and additional linear axes are
specifically excluded. Only contact probes are considered, both
solid and trigger types. The standard attempts to define the sim-
plest testing that can yield adequate results within a reasonable
time frame. More complete testing can be negotiated between
the supplier and user. The machine performance testing
includes the effective diameter test, the single point articulation
performance test and the volumetric performance test. Due to
the portability of these machines, there are numerous factors
that affect the relative productivity of measuring systems, which
include variables attributable to both the measurement system
and the workpiece. The standard does not address methods to
specify and evaluate productivity.
If there is agreement between the supplier and the user, the
standard allows for parts of the environmental test section to be
deferred or bypassed and only the performance tests to be per-
formed.
4.2 Section 2: Definitions
Section 2 includes not only a glossary, taken from B89.5 [6] and
the VIM [7], but also includes the AACMM Classification Form,
the Environmental Specification Form, and the Performance
Evaluation Form. Machines are classified by the number of rota-
tional joints in the shoulder, elbow and wrist. The more
common configurations are detailed, but there is also provision
to specify any special, nonstandard configuration. As well as the
number of joints, the number of degrees of rotation of each joint
is also specified, along with the encoder resolution. A typical
classification is shown in Fig. 1.
4.3 Section 3: Environmental Specifications
This section of the Standard contains five subsections, namely
3.1 General
3.2 Temperature and Humidity
3.3 Vibration
3.4 Electrical
3.5 Mounting Stiffness and Orientation
Section 3.1 states that the user has the responsibility for con-
ducting all of the environmental tests at their facility and pro-
REVIEW PAPERS
viding an environment in accordance with the supplier specified
requirements. It is emphasized that this is not just the thermal
environment but that particulates, oils, water, etc. can have a
detrimental effect on the machines performance, as well as
increasing friction in the joints and accelerating wear.
In Section 3.2, the opening paragraph discusses some of the
boiler-plate issues of taking measurements at a non-standard
temperature and how ISO/TR 16015 [4] deals with this. Two
conditions are addressed, either the measurements are taken at
20 C, or the thermal error index (TEI) is used as a reasonable
percentage of the working tolerance. The user is referred to
Appendix H for a detailed discussion on the four components
used to determine the TEI, namely, the nominal differential
expansion (NDE) error, the uncertainty of nominal differential
expansion (UNDE), the uncertainty due to temperature meas-
urement (UTM), and the environmental thermal variation error
(ETVE). Due to the portability of this class of machine, the
thermal degradation of any measurement should be considered
on a case-by-case basis. The paragraph concludes by noting that
the thermal environment is not limited to the environment
within the work zone, but also includes the environment that is
in contact with the machine itself. The standard also recom-
mends several good metrology practices, such as avoiding direct
sunlight and other direct radiant energy sources (such as light-
ing), which should not be closer to any part of the machine than
twice the nominal arm length.
The paragraph on vibration issues notes that unlike conven-
tional CMMs, performing a seismic analysis of the mounting
configuration is not generally possible with this class of
machine, primarily due to cost considerations due to the multi-
tude of mounting configurations. Practical applications range
from rigidly clamping the machine to a surface plate, to using
double-sided tape to secure it to the under side of a wing of a
Boeing 737. If vibration effects are considered a major issue, an
extensive appendix is included (Appendix C) that explains how
to conduct any seismic vibration verification testing that may be
required. This appendix is essentially the same as that in
B89.4.1. [2]
The paragraph on electrical requirements likewise, is essen-
tially the same as B89.4.1 [2], highlighting the fact that the user
is responsible for providing electrical power as specified by the
supplier.
Although it is not always considered as an environmental
attribute, a section on mounting stiffness and orientation is
included because the performance of the AACMM may be influ-
enced by the mounting orientation, and the stability and stiff-
ness of the installation. These machines allow any number of
mounting configurations to be used, but thought should be
given to the application. To illustrate this point, consider that
the machine is often evaluated while mounted on a surface
plate. For this configuration, the volume above the arm will typ-
ically not be evaluated, as shown by the dotted lines in Fig. 2(a).
Due to the mounting flexibility of this class of machine, it may
be that for ease of use, the machine is mounted as shown in Fig.
2(b). In this orientation, measurements will be taken within the
volume that had previously not been evaluated. The Standard
Figure 1. A typical machine of the 2-2-3 configuration, with A-B-
C-D-E-F-G rotation.
Figure 2 (a). AACMM mounted vertically for evaluation.
Figure 2 (b). AACMM mounted horizontally for use.
B
A
D
C
E
F
G
Arm doesnt get
evaluated here
Vertical axis
Region not
previously
evaluated
REVIEW PAPERS
strongly recommends that testing be performed in the orienta-
tion(s) in which the arm is to be used.
4.4 Section 4: Environmental Tests
The general format is that Section 4 has a one-to-one correlation
with Section 3, one defining the requirement, the other defin-
ing the specific test for that requirement. Paragraph 4.1 iterates
that the user is responsible for providing an acceptable environ-
ment for conducting the tests.
Thermal testing and vibration testing are discussed in general
terms, noting that due to the portability of these machines, care
should be exercised in the interpretation of the data and its appli-
cability to different locations and conditions. Tests are fully
described in Appendix B and Appendix C respectively. The stan-
dard states that electrical tests are, in general, an unwarranted
expense and should only be undertaken in the event that the
machine does not meet performance specifications and there is
reason to suspect the electrical power. If this is the case, detailed
electrical testing is discussed in Appendix D.
The maximum values for mounting requirements are speci-
fied by the supplier in the Environmental Specification Form.
These are forces along each of the three axes and torques about
each of the three axes. The user should check the interface for
translational forces and torsional moments. A suggested
methodology is shown in Figs. 3 (a), 3 (b) and 3 (c).
4.5 Section 5: Machine Performance
There are four subsections, one general and one for each of the
three suggested tests, as follows.
5.1 General
5.2 Effective Diameter Performance Test
5.3 Single Point Articulation Performance Test
5.4 Volumetric Performance Test
The Effective Diameter Performance Test is performed by
measuring a calibrated 10 to 50 mm diameter sphere three times
at a fixed location, using nine points per measurement, four on
the equator, four at a latitude of 45 and rotated 45 to the first
four and one on the pole. From the measured data, three diam-
eters are calculated using the AACMM software. These are com-
pared to the calibrated value and the largest difference
reported, regardless of the sign. This is a quick and easy test that
may reveal excessive hysteresis in the machine or probe,
improper calibration, inadequate mounting, or out of roundness
of the probe. If outliers are suspected, it is suggested that the
entire test be repeated. A suggested format for reporting the
results is shown in Table 1.
Whereas the Effective Diameter Test minimizes the articula-
tion, the single point articulation test is used to determine the
ability of the machine to provide similar results on measuring
the same feature. It should be noted that due to the inherent
design of AACMMs, this is not the same as a repeatability test.
For a hard probe, the center coordinates of the sphere are meas-
ured in a kinematic seat. Measurements are taken at three dif-
ferent locations within the volume, with the default values being
0 % to 20 % of the arm length, 20 % to 80 % of the arm length
and 80 % to 100 % of the arm length, not all on the same plane.
Figure 3 (a). Definition of terms.
Figure 3 (b). Measuring forces.
Figure 3 (c). Measuring torques.
X FX Forces (Along X)
Moments (About X)
Displacement Due to Force
Rotation Due to Torque
Force
MX
D
L
T
D
__
L
D
MZ
Z
FZ
Y
FY
MY
REVIEW PAPERS
Ten default articulations are also suggested, with five shown in
Fig. 4 and the other five with the wrist rotated 180.
From the ten sets of measured XYZ triplets for each location,
the average is calculated for each axis. Then,
(1)
where X
i
, Y
i
, Z
i
are the measured coordinates
X
a
, Y
a
, Z
a
are the average coordinates
and the maximum
i
is compared to the Maximum Deviation on
the Performance Specification Form for each location. Alterna-
tively (or as well as) the 2s
SPAT
value can be found from:
(2)
The Volumetric Performance Test is performed using two cal-
ibrated length artifacts, the default being a ball bar, one between
50 % to 75 % of the arm length and the other 120 % to 150 %
of the arm length. It is recommended that temperature correc-
tion (if available) or compensation be used.
For this test, the machine volume is divided into eight equal
octants, shown in Fig. 5.
The artifacts are positioned horizontally, vertically and at 45.
The default positions are near and far, with near being
less than 50 % of the arm length and far being greater than
50 % of the arm length. Positions are either radial or tangen-
tial. The default positions attempt to cover as much of the
volume as possible, recognizing practical restrictions imposed
by the long ball bar. Twenty default positions are suggested,
detailed in Table 2.
From the measured data, three statistics can be computed.
The maximum deviation, D
i
, is found from:
(3)
where L
i
is the measured value and L
CAL
is the value fromthe cal-
ibration certificate. The RANGE is computed fromthe maximum
deviation, D
MAX
, minus the minimum deviation, D
MIN
:
(4)
and two times the root-mean-square (RMS) can be found from:
(5)
where in this case n = 20.
i
=
(X
i
X
a
)
2
+ (Y
i
Y
a
)
2
+ (Z
i
Z
a
)
2
,
i
2
D
n
2 RMS = 2 ,
RANGE = D
MAX
D
MIN
,
D
i
= L
i
L
CAL
,
i
2
(n 1)
2s
SPAT
= 2 .
Y
X
Z
Figure 4. Isometric view of default locations.
Table 1. Suggested format to report effective diameter test
results.
Diameter
deviation, m
D
1
D
cal
D
2
D
cal
D
3
D
cal
Max Deviation
Octant #3
Front
Octant #4
Front
Octant #1
Front
Octant #2
Front
Octant #5
Front
Octant #6
Front
Octant #7
Front
Octant #8
Front
Figure 5. Octant numbering scheme.
REVIEW PAPERS
4.6 Section 6: Test Equipment
The content is consistent with other B89 Standards, emphasiz-
ing that all test equipment should have metrology attributes 1/5
(or for temperature, 1/10) of that being measured, i.e. the
sphere on a ball bar shall be spherical within 1/5 of the working
tolerance. Appendix F of the standard provides details on the
design, use and mounting of ball bars. Also included is a brief
description of some of the error sources including, mounting
configuration errors, thermal errors, hysteresis errors, probing
errors and the influence of vibration. Similarly, Appendix G of
the standard discusses design recommendations and consider-
ations for kinematic seats, including trihedral sockets, conical
seats and chamfered hole seats.
4.7 Appendices
The appendices provide a wealth of information to users who
want to maximize the benefits (and minimize the errors) that
can be obtained from this class of machine. While many of the
appendices have been discussed in the body of the paper, the
usefulness of the other appendices is briefly discussed here.
As with any piece of measuring equipment, it is generally
good practice to periodically conduct some form of interim
check. Many users believe that this becomes extremely impor-
tant with AACMMs, which can be reconfigured to use different
mounting locations and orientations. A detailed discussion is
included in Appendix E of the standard.
Appendix I of the standard discusses the statistics used to
specify the performance of this class of machine. Essentially, an
attempt was made to harmonize ANSI and ISO standards. For
example, the Single Point Articulation Performance Test
(SPAT) is reported as a maximum deviation (as the linear dis-
placement accuracy test, LDA, in ASME B89.4.1 (b)), as well as
a standard deviation as shown in the GUM. [8] The latter value
is more robust than the maximum deviation and is useful when
performing an uncertainty evaluation as it represents a Type A
uncertainty contributor.
Table 2. Recommended artifact positions.
Position
Artifact
Length
Octants Inclination Distance Direction
1 short 8 Horizontal Near Radial
3 short 1 & 2 Horizontal Far Tangential
4 short 4 & 7 45 degree Far Tangential
5 short 7 & 3 Vertical Far Tangential
6 short 1 & 5 Vertical Far Tangential
8 long 2 & 8 45 degree Near Tangential
10 long 1 & 7 45 degree Near Tangential
18 long 3 & 4 Horizontal Near Tangential
19 long 2 & 6 Vertical Far Tangential
20 long 4 & 8 Vertical Near Tangential
REVIEW PAPERS
Appendix J of the standard discusses the application of deci-
sion rules to AACMM performance evaluation. The standard
states:
As a result, ASME B89 standards have adopted the
general statement, B89 standards that adopt stan-
dards referencing ISO 14253-1 as a normative stan-
dard shall explicitly state a different default decision
rule, where the 4:1 simple acceptance and rejection
rule from B89.7.3.1 shall be the default rule unless a
different rule is specified.
This is included to reflect the typical practice in U.S. indus-
try of accepting tolerances up to the upper and lower specifica-
tion limits, rather than using the ISO 14253-1 [9] default
decision rule which states that the expanded uncertainty must
be subtracted from the specification limits.
5. Conclusions
The economic benefits of standardization have been well docu-
mented over the years. The primary purpose of ASME B89 stan-
dards is to facilitate commerce between the seller and purchaser
of metrological equipment. In most cases, defining a minimum
set of easy to perform and replicate standard tests enable the
seller and purchaser to agree on stated system parameters and
to avoid the need for expensive litigation. If a more detailed or
thorough testing plan is required, it can still be arranged by
mutual agreement between the two parties.
In general, ASME B89 standards do not address issues of
design or material selection, but instead focus on evaluation.
The documents are not strictly calibration procedures, but they
can and do form the basis of a calibration procedure, using in
most cases calibrated artifacts. Most ASME B89 documents
contain a significant amount of what is loosely defined as edu-
cation material, when compared to corresponding ISO docu-
ments. This additional material is usually contained in
non-mandatory appendices that are not part of the requirements
of the standard, but that are provided simply as additional infor-
mation.
The ASME B89 standards and technical reports represent a
suite of documents that provide guidance on how to use selected
metrology equipment in the most appropriate manner. Starting
with an evaluation of the equipment, the user is then provided
guidance on the development of a measurement plan (see
B89.7.2 [10]). For the selected application, estimates of the
uncertainty can be made (see B89.7.3.2) [11] as well as estab-
lishing and agreeing on the most appropriate decision rule (see
B89.7.3.1 [12]). For the more stringent requirements, a thor-
ough treatment on risk analysis is provided (see B89.7.4.1
[13]). This Technical Report is written to include the mathemat-
ical details, but also contains numerous charts that the casual
user can use to determine risk levels under different conditions.
This document can be used for both the calibration function and
for product acceptance on the shop floor. To complete the loop,
information is given on traceability to the SI unit of length,
which can be used as a basis for all other SI basic units (see
B89.7.5 [14]).
6. References
[1] ASME B89.4.22 Methods for Performance Evaluation of Artic-
ulated Arm Coordinate Measuring Machines, 2004.
[2] ASME B89.4.1 Methods for Performance Evaluation of Coordi-
nate Measuring Machines, 2002.
[3] ASME B89.6.2 Temperature and Humidity Environment for
Dimensional Measurement, 2003.
[4] ISO/TR 16015:2003(E), Geometrical product specifications
(GPS) Systematic errors and contributions to measurement
uncertainty of length measurement due to thermal influences,
ISO, Switzerland, 2003.
[5] G.W. Caskey, S.D. Phillips and B.R. Borchardt, Application of the
ASME B89.1.12M Standard to an Articulating Arm Coordinate
Measuring Machine, NISTIR 5562(R), National Institute of Stan-
dards and Technology, Gaithersburg, MD, 1994.
[6] ASME B89.5 Compendium of Definitions for Dimensional
Metrology from ASME Codes and Standards Documents, (unpub-
lished Technical Report).
[7] International Organization for Standardization, International
Vocabulary of Basic and General Terms in Metrology, (Second
Edition), ISO, Switzerland, 1993.
[8] International Organization for Standardization, Guide to the
Expression of Uncertainty in Measurement, 1995. (This document
is also available as the American National Standards Institute, U.S.
Guide to the Expression of Uncertainty in Measurement,
ANSI/NCSL Z540-2-1997.)
[9] ISO 14253-1 Geometrical Product Specifications (GPS)
Inspection by Measurement of Workpieces and Measuring Equip-
ment Part 1: Decision Rules for Proving Conformance or Non-
Conformance with Specifications, 1998.
[10] ASME B89.7.2 Dimensional Measurement Planning, 2004.
[11] ASME B89.7.3.2 Guidelines for the Evaluation of Dimensional
Measurement Uncertainty, 2007.
[12] ASME B89.7.3.1 Guidelines for Decision Rules: Considering
Measurement Uncertainty in Determining Conformance to Spec-
ifications, 2006.
[13] ASME B89.7.4.1 Measurement Uncertainty and Conformance
Testing: Risk Analysis, 2005.
[14] ASME B89.7.5 Metrological Traceability of Dimensional Meas-
urements to the SI Unit of Length, 2006.
Note: ASME Documents are published by: The American
Society of Mechanical Engineers, Three Park Avenue, New
York, NY, 10016-5990 USA.
ISO Documents are published by: International Organization
for Standardization (ISO), 1 rue de Varemb, Case postale 56,
CH-1121, Genve 20, Switzerland/Suisse CH- 1121.
REVIEW PAPERS
1. Introduction
It is hard to overestimate the impact that atomic timekeeping
has had on modern society. Many everyday technologies that
we often take for granted, such as cellular telephones, Global
Positioning System (GPS) satellite receivers, and the electric
power grid, rely upon the accuracy of atomic clocks. This makes
it easy to forget that the era of atomic timekeeping began rela-
tively recently; less than an average lifetime ago. The current
year (2007) marks the 40
th
anniversary of the redefinition of the
International System of Units (SI) second based on cesium
(1967). [1] Prior to 1967, the definition of the second had
always been based on astronomical time scales. The year 2008
marks the 50
th
anniversary of the first publication (1958) of the
definition of the atomic second, defined as 9,192,631,770
periods of the radiation of the ground state hyperfine transition
in cesium. [2] The year 2008 also marks the 60
th
anniversary of
the first prototype of an atomic clock, an ammonia maser that
was first demonstrated at the National Bureau of Standards
(NBS), later known as the National Institute of Standards and
Technology (NIST) in 1948. [3, 4] With these dates in mind,
lets begin with a brief look at some fundamental concepts and
early standards, and then move on to explore present-day
atomic oscillators.
1
Michael A. Lombardi, Thomas P. Heavner and Steven R. Jefferts
Abstract: As the base unit of time interval, the second holds a place of preeminence in the world of metrology. Time
interval, and its reciprocal, frequency, can be measured with more resolution and less uncertainty than any of the other
physical quantities. The precise realization of the second was made possible by the development of frequency standards
based on the energy transitions of atoms, an event that not only revolutionized the science of timekeeping, but also
forever changed many other areas of metrology. The definitions of other units in the international system (SI), includ-
ing the meter, candela, ampere, and volt, now depend upon the definition of the second. In the approximate half century
since atomic frequency standards were first introduced, the progress in time and frequency metrology has been rapid
and relentless. Today, the National Institute of Standards and Technology (NIST), maintains the worlds most accurate
primary frequency standard, a laser-cooled cesium fountain known as NIST-F1. Since its completion in 1998, NIST-F1
has been continuously improved and is currently accurate to within 4 10
-16
. NIST-F2, a second-generation atomic
fountain standard now being built, promises even better accuracy. This paper explores both the history and the tech-
nology of NIST primary frequency standards, beginning in the pre-atomic era and continuing through the present day.
Michael A. Lombardi
Thomas P. Heavner
Steven R. Jefferts
Time and Frequency Division
325 Broadway
Boulder, CO 80305-3328 USA
Email: lombardi@nist.gov
1
This paper is a contribution of the United States government and is
not subject to copyright. Commercial products are mentioned only for
the sake of technical and historical completeness, and this does not
imply endorsement by NIST.
REVIEW PAPERS
2. Fundamentals of Frequency
and Time
All time and frequency standards are ref-
erenced to a periodic event that repeats
at a nearly constant rate. The periodic
event is produced by a device called a
resonator. The resonator is driven by an
energy source, and taken together, the
energy source and resonator form an
oscillator. The oscillator runs at a rate
called the resonance frequency. The fre-
quency, f, is the reciprocal of the period
of oscillation, T; therefore f = 1/T. Con-
versely, the period, measured in units of
time interval, is the reciprocal of the fre-
quency: T = 1/f.
To illustrate this, consider the simple
example of a grandfather clock, where a
pendulum (and its energy source) serves
as the oscillator. The pendulum has a res-
onance frequency (f) of 1 Hz, and a
period (T) of 1 s. Thus, it is easy to see
the close relationship between time inter-
val and frequency. All clocks are refer-
enced to an oscillator, and any
uncertainty or change in the frequency of
the oscillator will result in a correspon-
ding uncertainty or change in the time-
keeping accuracy of the clock.
Throughout this paper, we express
oscillator performance as f / f, where f
is the combined uncertainty of the fre-
quency with respect to its nominal value,
and f is the nominal value of the fre-
quency. The quantity f includes any
offset of the frequency from its nominal
value, as well as any variation in the fre-
quency over time. Please note that when
primary cesium frequency standards are
discussed, we quote their performance in
terms of accuracy, rather than uncer-
tainty. This is because cesium oscillators
are intrinsic standards, and the second,
the base unit of time interval, is defined
based on the cesium resonance (see
Section 7). Therefore, by definition,
primary cesium frequency standards
serve as the ultimate measurement refer-
ences for time interval and frequency,
and are inherently accurate. Their sys-
tematic frequency biases (Type B uncer-
tainties) are periodically evaluated and
corrections are applied. The accuracy of
a primary cesium standard is limited by
the uncertainties associated with these
corrections. For purposes of interna-
tional comparison, these uncertainty esti-
mates have historically been reported by
NIST and other national metrology insti-
tutes with a coverage factor of one, so
k = 1 should be assumed while reading
this paper.
3. Early Standards
The quest to improve timekeeping is one
of mankinds oldest pursuits, and has
essentially been a search to find better
oscillators. The best oscillators have a
period that is well characterized, not
easily perturbed, and as stable as possi-
ble. Early astronomers quickly noted that
the Earths rotation on its axis served as
a natural oscillator, and the second was
long defined as a fraction (1/86 400) of
the length of the solar day. Later, because
the length of the day fluctuates by a large
amount seasonally, the second was more
accurately defined as a fraction of the
average length of the solar day (known as
the mean solar second). To create a more
stable unit of time interval, astronomers
later chose the period of the orbital
motion of Earth about the Sun (nomi-
nally 1 year) as the basis for the defini-
tion of the second. In 1956, the
ephemeris second (1/31 556 925.9747
of the tropical year 1900) was formally
adopted by the General Conference of
Weights and Measures (CGPM) as the SI
second. While the ephemeris second was
indeed more stable than the mean solar
second, it was impractical to measure,
and it was never available or used in
metrology laboratories or as a general
timekeeping standard. Its few applica-
tions were limited to the world of astron-
omy. [5]
In the pre-atomic era, mechanical and
electrical oscillators served as the labora-
tory standards for time interval and fre-
quency measurements. At NIST (then
known as NBS, or the National Bureau of
Standards), pendulum clocks originally
served as the standards of time interval,
while tuned circuits and later quartz
oscillators served as the standards of fre-
quency. Pendulum clocks were based on
the principles first outlined by Galileo
Galilei between 1637 and 1639. Galileo
had observed that a given pendulum took
the same amount of time to swing com-
pletely through a wide arc as it did a
small arc, and recognized that this
natural periodicity could be applied to
time measurement. Shortly after his
death in 1642, reliable mechanisms were
designed to keep the pendulum in
motion, thus making it possible to build
pendulum clocks. [6, 7] For nearly 300
years after Galileos discovery, pendulum
clocks dominated the world of high accu-
racy timekeeping. Their practical per-
formance limit was reached in the 1920s
when W. H. Shortt designed and manu-
factured a clock with two pendulums,
one a slave and the other a master. The
slave pendulum moved the clocks hands,
and freed the master pendulum of tasks
that would disturb its regularity. The
Shortt pendulum clock kept time to
within a few seconds per year (1 10
7
).
NBS used a pendulum clock manufac-
tured by Clemens Riefler of Munich,
Germany as the national time interval
standard from about 1904 to 1929, when
it was replaced by a Shortt pendu-
lum. [8] Figure 1 shows the Riefler clock
on display in the NIST museum in
Gaithersburg, MD, where a Shortt pen-
Figure 1. The Riefler pendulum clock,
which was the primary standard for time
interval measurements from 1904 to 1929,
is now on display at the NIST museum in
Gaithersburg, MD.
REVIEW PAPERS
dulum also is on display. Figure 2 show a 1928 advertisement
for the Shortt pendulum, which was billed by its makers as The
Perfect Clock.
The first frequency standards at NBS were developed begin-
ning in about 1911 to calibrate wavemeters and to support the
burgeoning radio industry. Frequency was obtained by calculat-
ing the resonance of an inductance-capacitance (LC) circuit.
The frequency uncertainty of these tuned circuits was eventually
reduced to less than 0.1 %. [8, 9] Quartz crystal oscillators,
based on the phenomenon of piezoelectricity discovered by P.
Curie in 1880, worked much better, resonating at a nearly con-
stant frequency when an electric current was applied. Credit for
developing the first quartz oscillator is generally given to Walter
Cady of Wesleyan University, who built laboratory prototypes
shortly after the end of World War I [10], and patented a piezo-
electric resonator designed as a frequency standard in 1923.
Quartz oscillators first appeared at NBS and other metrology
laboratories in the early 1920s, and were soon used to control
radio transmission frequencies. [11] By 1929, the NBS fre-
quency standard was a group of four 100-kHz quartz oscillators
that had been developed by Bell Telephone Laboratories (Fig.
3), with a frequency uncertainty of 1 10
-7
. [12] By 1950,
quartz standards reduced this uncertainty by a factor of 100;
producing frequency within 1 10
-9
. [13] During this period of
rapid advancement in oscillator technology, quartz oscillators
replaced pendulum clocks as the NBS standard for time inter-
val. [14] Thus, for the first time, the NBS reference signals for
measurements of both time interval and frequency could be
obtained from the same primary standard.
Quartz oscillators are now used for an almost limitless
number of applications. Billions of quartz oscillators are man-
ufactured annually for use inside clocks, watches, cell phones,
computers, radios, and nearly every type of electronic circuit.
Even so, quartz oscillators are not ideal frequency standards.
Their resonance frequency depends on the size and shape of the
crystal; and no two crystals can be exactly alike or produce
exactly the same frequency. In addition, the frequency of a
quartz oscillator changes slowly over time due to aging, and can
change more rapidly due to the effects of environmental factors,
such as temperature, humidity, pressure, and vibration. [7]
These limitations make them unsuitable for some applications,
and the quest for better accuracy led to the development of
atomic oscillators.
4. Fundamentals of Atomic Oscillators
Oscillators derived from resonant transitions in atoms or mol-
ecules had several advantages over the oscillators that preceded
them. An unperturbed atomic transition is identical from atom
to atom, so that, unlike a group of quartz oscillators, a group of
atomic oscillators should all generate the same frequency. Also,
unlike all electrical or mechanical resonators, atoms do not wear
out. Additionally, at least as far as we know, atoms do not
change their properties over time. [15] These features were
appreciated by Lord Kelvin, who suggested using transitions in
sodium and hydrogen atoms as timekeeping oscillators in 1879.
[16] However, it wasnt until the mid-20
th
century that technol-
ogy made his idea possible.
Atomic oscillators use the quantized energy levels in atoms
and molecules as the source of their resonance frequency. The
laws of quantum mechanics dictate that the energies of a bound
system, such as an atom, have certain discrete values. An elec-
tromagnetic field at a particular frequency can boost an atom
from one energy level to a higher one. Or, an atom at a high
energy level can drop to a lower level by emitting energy. The
resonance frequency (f
o
) of an atomic oscillator is the difference
between the two energy levels, E
1
and E
2
, divided by Plancks
constant, h:
(1)
The basic principle of the atomic oscillator is simple: Since all
atoms of a specific element are identical, they should produce
the exact same frequency when they absorb energy or release
energy. [17, 18]
Atomic oscillators provided a major breakthrough in both
accuracy and stability, easily surpassing the performance of all
previous standards. In theory, the atom is a perfect pendulum
whose oscillations can be used as a standard of frequency, or
counted to measure time interval. However, there are still some
fundamental, as well as practical, factors that can limit the sta-
bility and accuracy of atomic oscillators. Atoms absorb or emit
f
o
= .
E
2
E
1
h
Figure 2. A 1928 advertisement for a dual-pendulum Shortt clock.
REVIEW PAPERS
energy over a small frequency range sur-
rounding f
o
, not at f
o
alone. All other
parameters being equal, the stability of
an atomic oscillator is proportional to f
o
and inversely proportional to the small
spread f
a
of absorption frequencies sur-
rounding f
o
. This spread of frequencies is
known as the resonance width, or
linewidth. The ratio of the resonance fre-
quency to the resonance width is known
as the quality factor, Q, where:
(2)
Oscillators with higher Qs are poten-
tially more stable and accurate, so it is
desirable to increase Q, either by using an
atomic transition where f
o
is as high as
possible, or by making f
a
as narrow as
possible. [17] In practice, of course, f
o
is
an inherent property of the atom used in
a given device (it is the same for all
cesium oscillators, for example) and
cannot be changed. However, it is possi-
ble to narrow the resonance width by
increasing the observation time of the
atoms. To illustrate this concept, consider
that most atomic oscillators work by fre-
quency-locking an external oscillator
(usually quartz) to the resonance fre-
quency of the atomic transition. The
external oscillator emits electromagnetic
radiation that illuminates the atoms and
causes them to change their energy state.
The challenge then, is to tune the fre-
quency of the external oscillator so that it
matches f
o
. The uncertainty of this tuning
process is usually reduced if the atoms are
kept in the radiation field for a longer
period of time. Mathematically, the reso-
nance width, f
a
, is set by the period,
d
,
that the atoms spend interacting with the
external oscillator. The fractional width
of the resonance is given by:
(3)
From this equation, it can be seen that
the optimum performance is obtained by
making the interaction period,
d
, as long
as possible in order to narrow the reso-
nance.
There are many other effects that can
limit the stability and accuracy of atomic
oscillators. The motion of the atoms
introduces uncertainty by causing appar-
ent shifts in the resonance frequencies
(the Doppler effect). Similarly, collisions
between atoms can broaden the reso-
nance or cause the frequency to shift.
Stray electromagnetic fields (including
the ever-present thermal, or blackbody
radiation) can perturb the resonance fre-
quency and introduce potential errors.
[18, 19] Therefore, while atoms are
perfect pendulums in theory, there are
many design features that must be imple-
mented before an atomic oscillator can
achieve the lowest possible uncertainties.
5. First Atomic Oscillators
The first atomic oscillator experiments
began about sixty years after Lord
Kelvins original suggestion, during the
explosion of advances in quantum
mechanics and microwave electronics
that took place before, during, and
shortly after the World War II. [17] Most
of the basic concepts of atomic oscilla-
tors were developed by Isidor Rabi and
his colleagues at Columbia University in
the 1930s and 40s. [20, 21] Rabi had
informally discussed applying the molec-
ular beam magnetic resonance technique
as a time and frequency standard with
NBS scientists in 1939. However, the
research was mostly halted during World
War II, and he publicly discussed the
possibility of atomic clocks for the first
time in a lecture given to the American
Physical Society and the American Asso-
ciation of Physics Teachers on January
20, 1945, [22] and in a story published
in the New York Times (Fig. 4) the fol-
.
1
d
f
o
f
a
f
o
Q = .
f
o
f
a
Figure 3. This group of 100 kHz quartz crystal oscillators served as the U.S. national
primary frequency standard of radio frequency in 1929.
Figure 4. The first public suggestion of an
atomic clock, New York Times, January 21,
1945.
REVIEW PAPERS
lowing day. [23] It seems likely that Rabi
expected the atomic transition in cesium
(
133
Cs) to be the resonator for the first
atomic oscillator. [24] The first experi-
ments to measure the frequency of
cesium resonance had been performed in
1940 by Rabis colleagues at Columbia
University, who estimated the frequency
of the hyperfine transition in cesium as
9 191.4 megacycles. [25]
As fate would have it, however, the
worlds first working atomic oscillator
was not based on cesium atoms, but
instead used the 23.8 GHz inversion
transition in the ammonia molecule as its
source of resonance. The ammonia
device was developed at NBS by Harold
Lyons and his associates. It consisted of
a quartz crystal oscillator, electronically
stabilized by the ammonia absorption
line, and frequency dividers that pro-
duced a 50 Hz signal from the stabilized
quartz oscillator. Developed for use as a
frequency standard, the device (Fig. 5)
was first operated on August 12, 1948,
although it was not publicly demon-
strated until January 1949. The heart of
the system, an eight meter long wave-
guide absorption cell filled with
ammonia, is shown wrapped around the
analog clock mounted on top of the
equipment cabinets. The analog clock
itself was there only for cosmetic pur-
poses. Two versions of the NBS ammonia
standard were built, with estimated fre-
quency uncertainties of 1 10
-7
and
2 10
-8
. Work on a third version was
halted when it became apparent that
atomic beam techniques offered more
promise for frequency standard develop-
ment. [3, 4, 24, 26] Although the
ammonia oscillator actually failed to out-
perform the best quartz standards of its
time, it provided a glimpse of what the
future would bring, and was widely pub-
licized. Lyons was given a five-minute
interview by Edward R. Murrow over the
CBS Network on January 14, 1949 [27]
and features also appeared in Time,
Newsweek, Business Week, and else-
where. [24] Lyons was even mentioned
in a popular cartoon feature a few years
later (Fig. 6), although the drawing was
not of the ammonia device, but rather of
the cesium standard that would later
become known as NBS-1 (see Section 9).
During the same year (1949) when
Lyons introduced the ammonia fre-
quency standard, Norman Ramsey of
Harvard University provided a critical
improvement that has since been utilized
by all modern primary frequency stan-
dards. In the early work of Rabi, the
atomic resonance was interrogated with
one long microwave pulse. This provided
the needed long interaction time between
the atom and microwave field, but sub-
jected the output frequency of the stan-
dard to Doppler shifts and other
uncertainties. Ramsey greatly reduced
these problems by inventing the sepa-
rated oscillatory field method. This
method interrogates the atoms with two
short microwave pulses, separated by
some distance along the beam path.
Applying the oscillating field in two steps
had many benefits, and made it possible
to build much more stable and accurate
standards. It narrows the resonance
width, reduces the sensitivity to
microwave power fluctuations and mag-
netic fields by factors of 10 to 100 or
more, and essentially eliminates the
Doppler effect. [17, 21, 28, 29, 30]
6. First Cesium Oscillators
When the ephemeris second (see Section
3) became the SI second in 1957, work
on atomic frequency standards had
already been progressing for more than a
decade. By the early 1950s, work had
begun in several national laboratories,
notably the National Physical Laboratory
(NPL) in England and at NBS in the
United States, to build atomic frequency
standards based on cesium.
Cesium has several properties that
Figure 5. The first atomic frequency standard, based on the ammonia molecule (1949).
Inventor Harold Lyons is on the right; Edward Condon, then the director of NBS, is on the left.
REVIEW PAPERS
made it a good choice as the source of
atomic resonance for a primary fre-
quency standard. Somewhat similar to
mercury, cesium is a soft, silvery-white
ductile metal. It becomes a liquid at about
28.4 C (slightly higher than room tem-
perature). Cesium atoms are relatively
heavy (133 amu), and, as a result, they
move at a relatively slow speed of about
130 m/s at room temperature. This
allows cesium atoms to stay in the inter-
action zone longer than hydrogen atoms,
for example, which travel at a speed of
about 1600 m/s at room temperature.
Cesium also has a relatively high hyper-
fine frequency (~9.2 GHz) when com-
pared to atoms used in other atomic
oscillators, such as rubidium (~6.8 GHz)
and hydrogen (~1.4 GHz). [30]
By 1950, while work on the ammonia
standard was still ongoing, NBS had
begun building its first cesium frequency
standard. [31] The NBS group, led by
Harold Lyons and Jesse Sherwood, con-
structed a machine using Rabis magnetic
resonance technique, and Sherwood
reported the first successful observation
of the cesium microwave transition at the
1952 meeting of the American Physical
Society. [32] Shortly afterwards, the
device was modified to use the Ramsey
technique of separated oscillating fields,
which reduced the resonance width to
just 300 Hz. Based on these results,
Lyons predicted an eventual accuracy
capability of 1 10
-10
. [26, 33]
Unfortunately, the program at NBS to
develop a cesium frequency standard was
interrupted in 1953 for several years,
partially due to budget issues and par-
tially due to decisions made to focus on
other areas. Sherwood left NBS in 1953
and Lyons followed in 1955. [8, 27, 34]
The cesium standard was completely dis-
assembled and moved from Washington,
DC to the new NBS laboratories in
Boulder, Colorado in 1954. [27] The
device was eventually reassembled with
many new components and improved
electronics, but it was not until the
19581959 period that this original
cesium beam frequency standard was
used to routinely calibrate the frequen-
cies of the working standards and
became known as NBS-1 (see Section 8).
[26, 35]
Meanwhile, Louis Essen and his asso-
ciates at the National Physical Labora-
tory (NPL) in Teddington, England, had
placed a similar cesium beam apparatus
with a resonance width of 340 Hz and an
accuracy of 1 10
-9
into operation in
June 1955. [36] The NPL device became
the worlds first cesium standard to be
used on a regular basis for the calibration
of secondary working frequency stan-
dards. [26] And, as we shall see in the
next section, it was used to help redefine
the SI second.
7. Atomic Second
Ephemeris time was determined by
measuring the position of the Moon with
respect to several surrounding stars.
The best observations of the Moon had
been recorded by the United States
Naval Observatory (USNO), where a
sophisticated dual-rate Moon camera
had been designed and put into opera-
tion in 1952 by William Markowitz. [37]
Moon observations were a tedious prac-
tice, and it was immediately noted by Sir
Edward Bullard, the director of NPL,
that observations of the Moon over a
period of four years would be required to
determine ephemeris time with the same
precision as was achieved in a matter of
minutes by their new cesium standard.
[38] In June 1955, due in part to the fact
that there were no reliable atomic oscil-
lators then operating in the United
States, NPL and the USNO began coop-
erating in a joint program whose goal
was to determine the frequency of
cesium with respect to the ephemeris
second. [2, 37, 39] Cesium would
provide a physical second that could be
realized in laboratories and used for
other measurements.
For a period of almost three years,
beginning in mid-1955 and lasting
through early 1958, frequency compar-
isons were made between the USNO and
NPL standards. The USNO standard was
a quartz oscillator steered to ephemeris
time by applying corrections obtained
with the Moon camera; the NPL stan-
dard was their cesium oscillator, then
accurate to less than 5 10
-10
. [39] The
comparisons were made based on
common-view observations of signals
from several time signal broadcast sta-
tions, including NBS radio station WWV,
then located in Beltsville, MD, and the
British stations MSF and GBR. [37, 39]
After averaging and analyzing the results
of these comparisons, it was determined
that the transition frequency of cesium
was 9 192 631 770 Hz, with an uncer-
tainty of 20 Hz. The uncertainty was
limited not by the cesium standard, but
rather by the difficulty of measuring
ephemeris time. The calculation of the
Figure 6. A 1953 cartoon featuring the NBS atomic clock.
REVIEW PAPERS
cesium frequency was published by
Markowitz and Essen and their col-
leagues in August 1958. It applied to
cesium in a zero magnetic field, and to
ephemeris time in a given tropical year
(1957). [2]
The publication of the cesium fre-
quency made the fate of astronomical
time seem certain, but almost a decade
passed before the definition of the
second was changed. In 1967, the SI
second was redefined by the General
Conference of Weights and Measures
(CGPM) as:
the duration of 9 192 631 770
periods of the radiation corre-
sponding to the transition
between the two hyperfine
levels of the ground state of the
caesium 133 atom. [1]
The definition ushered in the newera of
atomic time. For the first time in history,
the second was no longer related to the
length of the day or the movement of the
planets, but instead to the intrinsic prop-
erty of an atom. This had an immediate
effect on timekeeping. Ephemeris time
soon faded into obscurity, but other astro-
nomical time scales were still used, and it
was decided that atomic time and astro-
nomical time should remain in relative
agreement. To keep the new atomic time
scale, known as Coordinated Universal
Time (UTC), within 0.9 s of an astro-
nomical time scale (UT1) based on the
mean solar second, the concept of leap
seconds was introduced. [37] From
19722007, 24 leap seconds have been
added to the UTC time scale. The addition
of a leap second essentially stops UTC for
one second so that UT1 can catch up.
As of 2007, the definition of the SI
second remains the same, except for a
slight amendment made in 1997. Calcu-
lations made by Wayne Itano of NBS in
the early 1980s [40] revealed that black-
body radiation can cause noticeable fre-
quency shifts in cesium standards, and
his work eventually resulted in an adden-
dum to the definition. The Comite Inter-
national des Poids et Mesures (CIPM)
affirmed in 1997 that the definition
refers to a cesium atom at rest at a ther-
modynamic temperature of 0 K. Thus, a
perfect realization of the SI second
would require the cesium atom to be in a
zero magnetic field in an environment
where the temperature is absolute zero
and where the atom has no residual
velocity. Therefore, as will be discussed
in Section 10, obtaining the best possible
realization of the SI second requires
measuring and estimating a number of
parameters, and then using the resulting
data to apply corrections to the output of
a primary frequency standard.
8. Commercial Cesium
Frequency Standards
Somewhat surprisingly, the worlds first
commercial cesium frequency standard
was introduced in October 1956, just
slightly more than one year after the
introduction of the NPL standard. The
commercial standard was manufactured
by the National Company of Malden, MA
and developed by a team led by Jerrold
Zacharias of the Massachusetts Institute
of Technology (MIT) [24], who had pre-
viously collaborated on early molecular
beam experiments with Rabi. [20] The
device was called the Atomichron, and at
least 50 were sold between 19561960.
Nearly all of these units were sold to the
United States military [24], but at least
two were in operation at NBS in
19581959, where they were compared
to the NBS standard to within parts in
10
10
. [35]
Cesium frequency standards entered
calibration and metrology laboratories in
the 1960s, primarily through the efforts
of the Hewlett-Packard (HP) Company,
who introduced their model 5060 stan-
dard in 1964. Much smaller than previ-
ous devices, such as the Atomichron, it
fit in a standard equipment rack. The
length of the Ramsey cavity was just
12.4 cm, and the device weighed less
than 30 kg. [41] By 1966, the HP 5060
had a specified accuracy of 1 10
-11
. [26].
The HP 5060 was followed by the HP
5061, which was manufactured from
1967 until the early 1990s, and by the
HP 5071, which was introduced in 1991.
The HP 5071 was microprocessor based,
and its internal firmware monitored and
controlled many of the parameters that
could change its frequency. This micro-
processor control, coupled with an
improved cesium beam tube, made it
more stable and accurate than it prede-
cessors. [42] The HP 5071 was later
manufactured by Agilent and is now
(2007) manufactured by Symmetricom,
Inc. It has a specified accuracy (with the
high performance beamtube) of 2 10
-13
,
and serves as a primary standard of fre-
quency at many national and private
metrology laboratories worldwide.
Figure 7 provides a simplified
schematic of a cesium beam frequency
standard. This design can be traced back
to the seminal work of Rabi and Ramsey.
Note that the design details of a cesium
beam standard can vary significantly
from model to model, but nearly all com-
mercial cesium oscillators employ basic
design principles similar to those
Figure 7. Diagram of a cesium-beam frequency standard using magnetic state-selection
and detection. The form of interrogation involves a U-shaped microwave cavity (the
Ramsey cavity) where the microwave interrogation fields are spatially separated.
Vacuum Cavity
State Selection Magnets
Microwave
Interrogation
Cavity
Frequency
Synthesizer
Quartz
Oscillator
Servo
Feedback
1 pps 5 MHz
9 192 631 770 Hz
Cesium
Oven
Getter
Getter
Detector
State Detection Magnets
REVIEW PAPERS
described here, as did all NBS/NIST
cesium primary frequency standards until
the 1990s.
Cesium is a complicated atom with
F = 3 and F = 4 ground states (see Fig. 8).
Each atomic state is characterized not
only by the quantum number F, but also
by a second quantum number, m
F
, which
can have integer values between F and
+F. Therefore, there are 16 possible mag-
netic states of cesium, but only the tran-
sition between the |4,0 and |3,0 states is
useful for a primary frequency standard,
because this transition is, to first order,
insensitive to magnetic fields. This |4,0
|3,0 hyperfine transition produces the
frequency used to define the SI second.
As shown on the left side of Fig. 7,
133
Cs atoms are heated to a gaseous state
in an oven. A beam of atoms emerges
from the oven at a temperature near
100 C and travels through a magnetic
field, where the beam is split into two
beams of atoms with different magnetic
states. One beam is absorbed by the
getter and is of no further interest, but
the other beam is deflected into the
microwave interrogation cavity (com-
monly known as the Ramsey cavity).
While inside the Ramsey cavity, the
cesium beam is exposed to a microwave
frequency from a quartz-based frequency
synthesizer. If this frequency is tuned to
precisely match the cesium resonance fre-
quency (9 192 631 770 Hz), some of the
atoms will change their magnetic state.
After leaving the Ramsey cavity, the
atoms pass through a second magnetic
field. These magnets direct only the
atoms that changed state to the detector;
the other atoms are directed to a getter
and absorbed. In essence, the magnets
located on both sides of the Ramsey
cavity serve as a gate that only allows
atoms that undergo the desired |4,0
|3,0 energy transition to pass through
and reach the detector. The detector
sends a feedback signal to a servo circuit
that continually tunes the quartz oscilla-
tor so that the maximum number of
atoms reach the detector, thereby increas-
ing the signal strength. This process is
analogous to carefully tuning a radio dial
until the loudest and clearest signal is
heard, and keeps the quartz oscillator fre-
quency locked as tightly as possible to
cesium resonance. [17, 18, 19, 30] Stan-
dard output frequencies, such as 1 Hz, 5
MHz, and 10 MHz, are then derived from
the locked quartz oscillator and used in
the laboratory as reference signals.
9. Cesium Beam Primary
Frequency Standards at
NBS/NIST
Seven different cesium beam devices
served as U.S. national primary fre-
quency standards (NPFS) at NBS/NIST
over an approximate 40 year period
(19591998). The standards were all
thermal cesium beam devices that oper-
ated by directing a beam of
133
Cs atoms
through a microwave cavity. They were
known as NBS-1 through NBS-6, and
NIST-7 (the agencys name was changed
from NBS to NIST in 1988), and dozens
of scientists, engineers, and researchers
contributed to their development. Some
key figures include Richard Mockler,
who oversaw the development of the first
three standards, Dave Glaze, who con-
tributed to six different standards (NBS-2
through NIST-7), and Bob Drullinger,
who led the development of NIST-7. This
section briefly describes each of the
cesium beam standards, and Table 1 pro-
vides a summary of their characteristics.
Figure 8. Schematic diagram of the cesium clock transition.
2
S
1/2
F = 3
F = 4
m
F
= 0
9,192,631,770 Hz
m
F
= 0
m
F
m
F
= 4
m
F
= 3
m
F
= 2
m
F
= 1
m
F
= 1
m
F
= 2
m
F
= 3
m
F
= 4
m
F
= 1
m
F
= 2
m
F
= 3
m
F
= 3
m
F
= 2
m
F
= 1
Table 1. Summary of NIST cesium beam primary frequency standards, including the best
published accuracy number.
*
NA indicates that NBS-4 was used in conjunction with NBS-5 and NBS-6, but was never offi-
cially designated as a standalone NPFS.
Cesium
Frequency
Standard
Lifetime as
Experimental
or
Operational
Device
Years of
Operation as
NPFS
Linewidth
Length of
Ramsey
Cavity
Best
Published
Accuracy
NBS1 19521962 19591960
300 Hz 55 cm 1 10
-11
[44]
NBS2 19591965 19601963
110 Hz 164 cm 8 10
-12
[44]
NBS3 19591970 19631970
48 Hz 366 cm 5 10
-13
[45]
NBS4 19651990s NA *
130 Hz 52.4 cm 3 10
-13
[46]
NBS5 19661974 19721974
45 Hz 374 cm 2 10
-13
[46]
NBS6 19741993 19751993
26 Hz 374 cm 8 10
-14
[48]
NIST7 19882001 19931998
62 Hz 155 cm 5 10
-15
[51]
REVIEW PAPERS
NBS-1 (Fig. 9) was the original cesium beam primary fre-
quency standard at NBS, previously described in Section 6. The
length of its Ramsey cavity was 55 cm and its linewidth was
about 300 Hz. [43] After being moved to Boulder, CO in 1954,
it was redesigned and eventually designated as the national stan-
dard for frequency in the spring of 1959, when its accuracy was
reported as 8.5 10
-11
. [35] On January 1, 1960, it was replaced
by NBS-2 as the NPFS, although it continued to be compared to
NBS-2 as a secondary standard until 1962, when it had a
reported accuracy of 1 10
-11
. Later that year it was converted
to an experimental thallium beam standard. [27, 44]
NBS-2 (Fig. 10) was designed to be much larger than NBS-1,
and the length of the Ramsey cavity was increased by a factor of
three, to 164 cm. As a result, the linewidth was narrowed to
about 110 Hz, and the accuracy was improved to 8 10
-12
. [44]
It was replaced by NBS-3 as the NPFS in September 1963, but
continued to be compared to NBS-3 until October 1965. Like its
predecessor NBS-1, it was converted to an experimental thallium
beam standard after being removed from service. [45]
Work on NBS-3 (Fig. 11) began in 1959. The length of the
Ramsey cavity was increased to 366 cm, more than twice as long
as NBS-2, and the linewidth was narrowed to just 48 Hz, result-
ing in a Q factor of 2 10
-8
, which was considered phenome-
nal for that era. [27] Its originally published accuracy was 5
10
-12
, but a series of modifications improved the accuracy by an
order of magnitude, reaching 5 10
-13
by 1969. [45] NBS-3
continued to serve as the NPFS until 1970, when it was com-
pletely dismantled. Parts of its vacuum system were later used in
NBS-5. [46] During the period from 1970 to 1972 no primary
standard was operational at NBS, and commercial cesium stan-
dards served as the NPFS while work on NBS-4 and NBS-5 was
underway.
NBS-4 (Fig. 12) was the smallest of the NBS/NIST primary
frequency standards, and had the longest operating life. Origi-
nally known as NBS-X4, it was built as part of a joint effort
between NBS and the Hewlett-Packard Company, and was not
originally intended to be used as a primary frequency standard.
Figure 9. NBS-1.
Figure 10. NBS-2.
Figure 12. NBS-4.
Figure 11. NBS-3.
REVIEW PAPERS
Work on this project began in late 1965, but NBS-4 was not
operational until early 1973. [46] NBS-4 was never officially
announced as the standalone NPFS. Instead, its role was to
serve in conjunction with NBS-5, and later NBS-6, either as a
comparison standard or as the primary standard when those
devices were not operational. Unlike the other primary standards,
which typically ran during the calibrations of secondary standards
only, NBS-4 often ran continuously for long periods of time. It had
a linewidth of about 130 Hz, the length of its Ramsey cavity was
52.4 cm[47], and its reported accuracy was about 3 10
-13
. [46]
NBS-4 remained operational until the early 1990s, and its beam
tube is now on display in the NIST museum in Boulder, CO.
The next two standards, NBS-5 and NBS-6 (Figs. 13 and 14),
served as the NPFS for a combined total of more than 20 years.
They were really the same standard; NBS-5 became known as
NBS-6 after some extensive modifications. Work on NBS-5 began
in 1966, and it became the NPFS in January 1973, more than
two years after NBS-3 was dismantled. It was a huge machine;
the Ramsey cavity was 374 cm in length, and the length of the
entire device was about 6 m. The reported accuracy was 1.8
10
-13
. [46] Significant modifications to NBS-5 began in March
1974, and the linewidth was narrowed from 45 Hz to 26 Hz.
The improvement was a direct result of more effective filtering
of high velocity atoms, the reduced scattering of low velocity
atoms, and the reduced velocity of the atomic beam. The
redesigned standard was renamed NBS-6, and became opera-
tional in March 1975. [47] NBS-6 eventually reached an accu-
racy of 8 10
-14
[48], and served as the NPFS until it was
replaced by NIST-7 in 1993. [49]
NIST-7 (Fig. 15) was the last thermal beam frequency stan-
dard developed at NIST, and was very different from its prede-
cessors. It used the newly available narrow linewidth lasers for
state selection and detection, thus replacing the magnets and
detectors (see Fig. 7) found in the earlier standards. Using light
instead of magnets had many advantages. Unlike magnetic
selection, which merely filtered out atoms in the wrong energy
state, the lasers optically pumped as many atoms as possible
into the desired energy state. This produced far more atoms and
generated a much stronger signal. [17] Work on NIST-7 began
Figure 14. NBS-6.
Figure 13. NBS-5.
Figure 15. NIST-7.
REVIEW PAPERS
in 1988. The original goal was to build a standard that could
provide 1 10
-14
accuracy [50], or timekeeping accuracy of
about 1 nanosecond per day. This goal was exceeded by a factor
of two, with the standard eventually reaching an accuracy of
5 10
-15
. The use of optical state detection enabled NIST-7 to
be nearly 20 times more accurate than NBS-6, in spite of the fact
that the length of the Ramsey cavity was just 155 cm, and the
resonance width was limited to 62 Hz. [19, 51] NIST-7 was
replaced as the NPFS in late 1998 by NIST-F1 (see Section 10),
but remained operational for a few years afterwards. It now
resides in the NIST museum in Boulder, CO.
10. Cesium Fountain Primary Frequency Standards
NIST-7 and other optically pumped thermal beam standards
were far more accurate than their predecessors, but their per-
formance was still fundamentally limited by the high velocity of
the cesium atoms and the resulting short interaction time. Foun-
tain standards were designed specifically to increase the inter-
action time. The concept of a cesium fountain was first
introduced by Zacharias in the 1950s, shortly before he became
involved in the development of the commercial Atomichron
standard. [21, 22, 24] His idea was simply to build a vertical
cesium beam standard with one Ramsey interaction zone. This
would allow slow atoms from the cesium oven to pass through
the interaction zone while traveling upward, to stop and reverse
their direction under the influence of gravity, and to pass
through the same interaction zone while traveling downward.
The two interactions with the microwaves reproduced Ramseys
two-pulse interaction scheme, and a ballistic flight traveling
only a meter upwards would increase the interaction time to
nearly 1 s, instead of the 10 ms interaction that was typical of
cesium beam devices. Unfortunately, Zacharias idea was pre-
mature. Because of collisions between the cesium atoms in the
beam, no signal was ever seen in his device. Essentially, all of the
slow atoms in the beam were scattered out of the beam by the
fast atoms that overtook them. [30, 52]
Zacharias idea was resurrected in the late 1980s by Steven
Chu and his colleagues at Stanford University, who built the
worlds first working fountain standard. Chus group first built
fountains using sodium atoms [53] and later using cesium [54],
although neither device was used as a primary frequency stan-
dard. Researchers at the Bureau National de Mtrologie Sys-
tmes de Rfrence Temps Espace (BNM-SYRTE) in Paris,
France later built the first primary frequency standard based on
Zacharias fountain concept. [55] Since then, many other
researchers at metrology laboratories around the world have
built, or are building, laser cooled cesium fountain standards.
The laser cooling of atoms [56] is the key to making a cesium
fountain work successfully. Laser cooling was proposed in 1975
[57], and first demonstrated by Dave Wineland and his col-
leagues at NBS in 1978. [58] It can be thought of as refrigera-
tion; the laser beam is a sink at very low entropy that interacts
with a sample with much higher entropy (the atomic sample).
Entropy is transferred from the atoms to the light field, via
optical interactions between the atom and the light field. The
entropy of the light field is raised (the atom scatters many
photons out of the laser beam with random direction and
phase), while the entropy of the atomic sample is lowered.
There are numerous laser cooling techniques, but cesium foun-
tains generally implement a scheme known as optical molasses.
This technique exerts a damping force on the atoms by using
three pairs of identical oppositely directed lasers (Fig. 16). The
lasers are tuned to a frequency slightly below the optical reso-
nance of the atoms. Atoms at the intersection of the six laser
beams are cooled to a temperature of 1 K or below in a few
hundred milliseconds. As if they were moving through thick
molasses, the cold cesium atoms slow down to about 1 cm/s, as
opposed to their ~100 m/s velocity at room temperature. This
allows a large sample or ball of atoms to be gathered together
and confined in one place. As an alternative to the thermody-
namic view presented earlier, this type of laser cooling can also
be thought of as the mechanical effect of light on the atom. If
the laser is tuned slightly lower in frequency than the optical res-
onance, the atom will preferentially absorb photons from the
laser beam toward which it is moving, as a result of the Doppler
effect. Each photon absorbed by the atom carries momentum in
the opposite direction of the atomic motion. The atom reemits
this photon in a random direction and, because the laser is
tuned below resonance, the atom reemits slightly more energy
than it absorbed (the atom reemits at the resonance frequency).
This cycle of absorbing a photon of slightly lower energy than
the reemitted photon is repeated many times per second (~ 10
7
)
and provides the basic laser-cooling cycle.
After less than three years of development, a cesium fountain
Figure 16. Simplified schematic diagram of a cesium fountain that
uses laser cooling.
REVIEW PAPERS
called NIST-F1 became the NIST primary frequency standard in
late 1998. [59] Designed by a team led by Steve Jefferts, NIST-
F1 (Fig. 17) was the first operational primary frequency stan-
dard at NIST to use the fountain technique and also the first to
use laser cooling, although an earlier experimental laser-cooled
fountain [60] had been built at NIST. Laser cooling made it pos-
sible for NIST-F1 to observe or measure atoms for a much
longer interval than cesium beam standards, and as a result, to
quickly surpass their accuracy.
The basic operation of NIST-F1 proceeds in a sequence of
steps: First, a sample of 10
8
cesium atoms with a volume of
approximately 1 cm
3
are laser cooled at the intersection of the six
laser beams (see Fig. 16). The temperature of this sample of
atoms is reduced to about 0.5 K in a few hundred milliseconds.
The atoms are then launched upwards at approximately 4 m/s
by detuning the frequency of the up and down laser beams to
make a moving optical molasses. The laser light is then extin-
guished by shutters so that no laser light interacts with the cesium
atoms along their ballistic flight path. The cloud of launched
cesium atoms, about 1 cm in diameter, is typically in the F = 4
ground state, but all m
F
levels are populated. The ball of cesium
atoms is next state-selected with a short microwave pulse that
drives the |4,0 atoms into |3,0 and leaves the other F = 4 atoms
unperturbed. The remaining F = 4 atoms are removed from the
cloud with a short optical blast. At this point the remaining
cesium atoms, all in the |3,0 state, enter the microwave cavity
(Fig. 16) with a velocity of around 3 m/s. The passage through the
cavity on the way up provides the first pulse of the two-pulse
(Ramsey) microwave interrogation sequence. The atoms reach
apogee about 1 m above the microwave cavity and begin to fall
due to gravity. On the way down, the atoms pass through the
cavity a second time, about 1 s after their first passage (other
fountains have interaction times near 0.5 s). The atoms are
detected optically with a laser tuned to the F = 4 r F = 5 optical
transition, similar to the detection process used in an optically
pumped beam standard such as NIST-7. [30]
The increased interaction time allows fountain-based fre-
quency standards to be more accurate than cesium beam stan-
dards. The fountain arrangement of NIST-F1 results in a
linewidth, f
a
, of ~ 1 Hz, much narrower than that of any pre-
vious NPFS. The Q factor is about 10
10
. Several evaluations of
the accuracy of NIST-F1 have been published [59, 61, 62], and
its present accuracy is 4 10
-16
. Evaluations of other fountain
frequency standards have reported nearly equivalent perform-
ance. [63-65]
When frequency standards reach accuracies measured in
parts in 10
16
, the limiting factors become fundamental in
nature. The accuracy of NIST-F1 is limited by two distinct
effects, a blackbody shift [62] and a density shift [61]. The
blackbody shift is simply the result of the cesium atoms inter-
acting with the thermal radiation emitted by the walls of the
300 K vacuum enclosure. The magnitude of this shift is large
(~2 10
-14
), but it can be corrected in the NISTF1 design with
an uncertainty as small as 2.6 10
-16
, a figure that corresponds
to an uncertainty of 1 K in the temperature of the thermal radi-
ation. Removing this limit will require either a great deal of the-
oretical calculation to develop an improved theory of
understanding the blackbody shift, or developing a cryogenic
vacuum system to reduce the magnitude of the effect. The
second possibility is now being pursued at NIST.
The density shift is caused by collisions between the cesium
Figure 17. The cesium fountain standard NISTF1.
REVIEW PAPERS
atoms in the launched sample, and is
quite large, as much as 4.0 10
-16
in
NIST-F1, with an uncertainty in the cor-
rection of 1.0 10
-16
. There are many
proposals for lowering the uncertainty of
the density shift correction, but it
remains a problem in the present gener-
ation of cesium fountains.
Another fundamental limit is pre-
sented by the gravitational redshift, an
effect that shifts the cesium frequency
with respect to its elevation above sea
level (by approximately 10
16
per meter).
A frequency correction for the gravita-
tional redshift is applied to NIST-F1. The
uncertainty of this correction, which is
about 3 10
-17
, is now insignificant, but
will be of more concern as future stan-
dards become more accurate. [66]
11. Second Generation
Cesium Fountain
A new cesium fountain, called NIST-F2,
is now under construction at NIST.
NIST-F2 is being designed to minimize
the uncertainties of both the blackbody
shift and the density shift. The blackbody
frequency shift [40] has lately been the
source of some controversy, with differ-
ent groups calculating different fre-
quency biases for the shift. [67, 68, 69,
70] Even if one of these competing
results is shown to be correct, the uncer-
tainty of the correction for the blackbody
frequency shift will still be limited to
about 1 10
-16
. In NIST-F2, a cryogenic
(T = 77 K) vacuum structure is being
built that includes the microwave cavi-
ties and flight tube above them. This
cryogenic vacuum system is expected to
reduce the magnitude of the blackbody
shift by a factor of about 250, from about
2 10
-14
to about 8 10
-17
. Thus, the
blackbody frequency shift will no longer
be a major source of uncertainty.
The density shift in NIST-F2 will be
significantly reduced by the use of a
clever idea theoretically developed at the
Istituto Nazionale di Ricerca Metrologica
(INRM) in Italy and experimentally
demonstrated at NIST. [71, 72] Multiple
balls of laser-cooled cesium atoms, as
many as 10 in NIST-F2, are launched in
quick succession (see Fig. 18) with the
first ball of cesium atoms having the
highest apogee, while the second ball has
an apogee just below the first, etc., so
that the trajectories of all the balls inter-
sect in the detection region. This reduces
the average cesium density by about a
factor of 10, but the signal to noise ratio
is preserved, as the same number of
atoms is finally detected. This technique
should reduce the uncertainty of the
density shift by slightly more than a
factor of three, to approximately 3 10
-17
.
As a result of these design improve-
ments, NIST-F2 is expected to eventually
reach an accuracy of < 1 10
-16
, limited
by effects due to the microwave fre-
quency of the oscillator. It appears likely
that NIST-F2 will be the last of the
cesium primary frequency standards at
Figure 18. The multiple ball (in this case 10) toss scheme. The horizontal axis in the
diagram is the time axis, while the vertical axis is the height above the launch region.
0.0 0.2 0.4 0.6 0.8 1.0 1.2
t [s]
z
[
m
]
2.0
1.5
1.0
0.5
0.0
Figure 19. The uncertainty of cesium primary frequency standards developed at NIST as a
function of the year they were introduced.
1940 1950 1960 1970 1980 1990 2000 2010 2020
Year
NBS-1
NBS-2
NBS-3
NBS-4
NBS-5
NBS-6
NIST-7
NIST-F1
NIST-F2
F
r
a
c
t
i
o
n
a
l
F
r
e
q
u
e
n
c
y
U
n
c
e
r
t
a
i
n
t
y
1E-9
1E-10
1E-11
1E-12
1E-13
1E-14
1E-15
1E-16
1E-17
REVIEW PAPERS
NIST, and that, in the future, better per-
formance will be achieved with optical
frequency standards.
Figure 19 shows the frequency uncer-
tainty of all of the cesium primary fre-
quency standards developed at NIST,
from NBS-1 to the anticipated perform-
ance of NIST-F2, plotted as a function of
the year they were introduced. The slope
of the line represents a reduction in
uncertainty of about one order of magni-
tude per decade.
12. Role of NIST-F1 in the
Realization of the SI Second
In addition to the primary standard
NIST-F1, two other systems are needed
to generate the SI second. The first
system is a continuously running time
scale that generates pulses every second
in real-time. The second system employs
satellite-based measurement techniques
to compare the NIST standards to the
other standards of the world. The results
of these comparisons are then submitted
to the Bureau International des Poids et
Mesures (BIPM) in France, which uses
them to help compute a post-processed
paper time scale known as Coordi-
nated Universal Time (UTC).
The NIST time scale, or UTC(NIST),
is an ensemble of about 10 oscillators
(commercial hydrogen masers and
cesium beam standards) that are contin-
uously compared to each other. NIST-F1
does not run continuously, but instead is
periodically used to measure and cali-
brate the frequency of the ensemble.
Although the hydrogen masers are not
inherently accurate, they are very stable,
and, once calibrated with NIST-F1, they
can maintain the accuracy of the time
scale when NIST-F1 is not operational.
The time scale output is steered based on
a weighted averaged of all of the oscilla-
tors in the ensemble, with the most
stable oscillators receiving the most
weight. [73] The time scale continuously
generates and outputs both a 5 MHz sine
wave frequency, and 1 pulse per second
timing signal, which is a physical, real-
time realization of the SI second.
UTC(NIST) serves as the standard of
time interval, frequency, and time-of-day
for the United States. It is distributed to
the public (at various levels of uncer-
tainty) through radio stations WWV,
WWVH, and WWVB, through a variety
of computer time services, and also to
paying customers who subscribe to NIST
remote calibration services. [74]
The comparison data sent to the BIPM
are collected from common-view obser-
vations of the GPS satellites, or by using
a geostationary satellite as a relay station,
a technique known as two-way satellite
time transfer. The BIPM uses the meas-
urement results to compute two time
scales, International Atomic Time (TAI),
which is used internally by the BIPM and
does not account for leap seconds, and
UTC, which is simply TAI corrected for
leap seconds. [5, 75]
The BIPM uses measurements from
primary frequency standards to deter-
mine the accuracy of UTC, and the meas-
urements from the other standards
(mostly commercial cesium beams and
hydrogen masers) to determine the sta-
bility of UTC. [76] As a primary stan-
dard, NIST-F1 contributes to the
accuracy of UTC. Since its completion in
1998, NIST-F1 has undergone about 20
accuracy evaluations (calibrations) that
have been submitted to the BIPM. An
accuracy evaluation usually requires
NIST-F1 to run continuously for about
20 to 40 days. The actual length of the
calibration procedure is typically deter-
mined by the noise of the satellite time
transfer process that is used to relay the
measurements to the BIPM.
Since 2004, five laboratories other
than NIST have operated cesium primary
frequency standards that have con-
tributed to the accuracy of UTC. These
laboratories include BNM-SYRTE,
INRM, NPL, the Physikalisch-Technische
Bundesanstalt (PTB) in Germany, and
the National Metrology Institute of Japan
(NMIJ). An additional group of about
300 oscillators from more than 50
national metrology institutes and astro-
nomical observatories contribute to the
frequency stability (and reliability) of
TAI and UTC. This group of oscillators
includes the commercial standards in the
NIST time scale. As is the case with
UTC(NIST), a weighted average is used,
and the most stable oscillators receive
the most weight. However, to keep the
most stable oscillators from dominating
the time scale, a maximum weight is
assigned for every period of calculation.
The maximum weight assigned to an indi-
vidual oscillator goes down as the number
of participating oscillators goes up.
As mentioned earlier, TAI and UTC are
post processed time scales, and therefore
do not produce any physical signals that
can be used in a metrology laboratory.
They are distributed only on paper,
through a document called the Circular-T
(published monthly by the BIPM and
available at www.bipm.org/en/scien-
tific/tai/). The results are presented in
the form of differences between local
time scales and UTC, for example as
UTC(NIST) UTC. The results reported
in the Circular-T verify that the measure-
ments made at NIST and other partici-
pating laboratories are traceable to UTC
and to the SI second. [75]
13. Summary and Conclusions
The history of atomic oscillators, from
the late 1940s to the present era, shows
a steady improvement in accuracy: from
about 1 10
-10
in the 1950s to less than
1 10
-15
today (2007). The present state
of the art in cesium primary standards is
defined by the cesium fountain NIST-F1,
which has achieved an accuracy of about
4 10
-16
. The NIST-F2 fountain now
under development will likely produce
better results, with an accuracy of less
than 1 10
-16
projected before the year
2010.
Atomic oscillators, currently based on
atomic microwave transitions, will even-
tually be replaced by devices based on
optical transitions, and the SI second will
likely be redefined. Atomic oscillators
using optical transitions have a much
higher resonance frequency (~10
15
Hz)
and are potentially accurate to less than
1 10
-17
. Optical frequency standards
are being investigated in many laborato-
ries worldwide with extremely promising
results. [15, 18, 76, 77] However, they
are still some years away from replacing
cesium devices as the worlds time and
frequency standards.
14. Acknowledgements
The authors thank the Measure review-
ers and NIST staff members Rich Fox,
John Kitching, John Lowe, and David
Smith for their helpful comments regard-
ing this manuscript.
REVIEW PAPERS
15. References
[1] Resolution 1 of the 13th Conference Generale des Poids et
Mesures (CGPM), 1967.
[2] W. Markowitz, R. Glenn Hall, L. Essen and J.V.L. Parry, Fre-
quency of Cesium in Terms of Ephemeris Time, Physical Review
Letters, vol. 1, no. 3, pp. 105-107, August 1958.
[3] The Atomic Clock: An Atomic Standard of Frequency and Time,
National Bureau of Standards Technical News Bulletin, vol. 33,
no. 2, pp. 1724, February 1949.
[4] H. Lyons, The Atomic Clock, Instruments, vol. 22, pp.
133135, December 1949.
[5] C. Audion and B. Guinot, The Measurement of Time: Time, Fre-
quency and the Atomic Clock, Cambridge University Press, Cam-
bridge, 2001.
[6] D. Christianson, Timepieces: Masterpieces of Chronometry,
Firefly Books Ltd., Toronto, Ontario, 2002.
[7] F.G. Major, The Quantum Beat: The Physical Principles of Atomic
Clocks, Springer-Verlag, New York, 1998.
[8] D. Sullivan, Time and Frequency Measurements at NIST: The
First 100 Years, Proceedings of the 2001 IEEE Frequency Control
Symposium, pp. 417, June 2001.
[9] J. Dellinger, Reducing the Guesswork in Tuning, Radio Broad-
cast, vol. 3, pp. 241245, December 1923.
[10] W. Cady, The Piezo-Electric Resonator, Proceedings of the IRE,
vol. 10, pp. 83114, April 1922.
[11] A. Hund, Uses and Possibilities of Piezoelectric Oscillators, Pro-
ceedings of the IRE, vol. 14, pp. 447469, December 1926.
[12] New Fundamental Frequency Standard, Technical News Bulletin
of the Bureau of Standards, no. 157, p. 43, May 1930.
[13] J. Shaull, Adjustment of High Precision Frequency and Time Stan-
dards, Proceedings of the IRE, vol. 38, pp. 615, January 1950.
[14] W. George, Production of Accurate One-Second Time Intervals,
Journal of Research of the National Bureau of Standards, vol. 21,
pp. 367373, September 1938.
[15] J. Bergquist, S. Jefferts, D. Wineland, Time Measurement at the
Millenium, Physics Today, vol. 54, pp. 3742, March 2001.
[16] W.F. Snyder, Lord Kelvin on Atoms as Fundamental Natural
Standards (for Base Units), IEEE Transactions on Instrumenta-
tion and Measurement, vol. IM-22, p. 99, March 1973.
[17] W. Itano and N. Ramsey, Accurate Measurement of Time, Scien-
tific American, vol. 269, pp. 5665, July 1993.
[18] S. Diddams, J. Bergquist, S. Jefferts, and C. Oates, Standards of
Time and Frequency at the Outset of the 21st Century, Science,
vol. 306, pp. 13181324, November 19, 2004.
[19] D. Sullivan, J. Bergquist, J. Bollinger, R. Drullinger, W. Itano, S.
Jefferts, W. Lee, D. Meekhof, T. Parker, F. Walls, and D. Wineland,
Primary Atomic Frequency Standards at NIST, Journal of
Research of the National Institute of Standards and Technology,
vol. 106, pp. 4763, January-February 2001.
[20] I. Rabi, J. Zacharias, S. Millman, and P. Kusch, A New Method
of Measuring Nuclear Magnetic Moment, Physical Review, vol.
53, p. 318, February 1938.
[21] N. Ramsey, Molecular Beams, Clarendon Press, Oxford, 1956.
[22] N. Ramsey, History of Atomic Clocks, Journal of Research of the
National Bureau of Standards, vol. 88, pp. 301320, September-
October 1983.
[23] W. Laurence, Cosmic Pendulum for Clock Planned, New York
Times, p. 34, January 21, 1945.
[24] P. Forman, Atomichron: The Atomic Clock from Concept to
Commercial Product, Proceedings of the IEEE, vol. 73, pp.
11811204, July 1985.
[25] S. Millman and P. Kusch, On the Radiofrequency Spectra of
Sodium, Rubidium, and Caesium, Physical Review, vol. 58, p.
438445, September 1940.
[26] R. Beehler, A Historical Review of Atomic Frequency Standards,
Proceedings of the IEEE, vol. 55, pp. 792805, June 1967.
[27] W. Snyder and C. Bragaw, Achievement in Radio: Seventy Years
of Radio Science, Technology, Standards, and Measurement at the
National Bureau of Standards, National Bureau of Standards
Special Publication 555, October 1986.
[28] N. Ramsey, A Molecular Beam Resonance Method with Sepa-
rated Oscillating Fields, Physical Review, vol. 78, pp. 695699,
June 1950.
[29] N. Ramsey, History of Early Atomic Clocks, Metrologia, vol. 42,
pp. S1S3, June 2005.
[30] S. Jefferts, T. Heavner, and E. Donley, Cesium Primary Frequency
References, Japanese Journal of Applied Physics, vol. 43, pp.
28032807, May 2004.
[31] H. Lyons, The Atomic Clock: A Universal Standard of Frequency
and Time, American Scholar, vol. 19, pp. 159168, April 1950.
[32] J. Sherwood, H. Lyons, R. McCracken, and P. Kusch, High fre-
quency lines in the hfs spectrum of cesium, Bulletin of the Amer-
ican Physical Society, vol. 27. p. 43. 1952.
[33] H. Lyons, Spectral lines as frequency standards, Annals of the New
York Academy of Sciences, vol. 55, pp. 831871, November 1952.
[34] P. Forman, The first atomic clock program: NBS, 1947-1954,
Proceedings of the 1985 Precise Time and Time Interval Meeting
(PTTI), pp. 117, 1985.
[35] R. Mockler, R. Beehler, and J. Barnes, An evaluation of a cesium
beam frequency standard, National Bureau of Standards Report
6075, October 1959.
[36] L. Essen and J. Parry, An Atomic Standard of Frequency and Time
Interval: A Caesium Resonator, Nature, vol. 176, pp. 280281,
August 1955.
[37] T. Jones, Splitting the Second: The Story of Atomic Time, Institute
of Physics Publishing, Bristol and Philadelphia, 2000.
[38] E. Bullard, An Atomic Standard of Frequency and Time Interval:
Definition of the Second of Time, Nature, vol. 176, p. 282,
August 1955.
[39] S. Leschiutta, The definition of the atomic second, Metrologia,
vol. 42, pp. S10S19, June 2005.
[40] W. Itano, L. Lewis, D. Wineland, Shift of
2
S
1/2
hyperfine split-
tings due to blackbody radiation, Physical Review A, vol. 25,
pp.12331235, February 1982.
[41] A. Bagley and L. Cutler, A Modern Solid-State Portable Cesium
Beam Frequency Standard, Proceedings of the 18th Annual Fre-
quency Control Symposium, pp. 344365, 1964.
[42] L. Cutler, Fifty years of commercial caesium clocks, Metrologia,
vol. 42, pp. S90S99, June 2005.
[43] R. Mockler, R. Beehler, and C. Snider, Atomic Beam Frequency
Standards, IRE Transactions on Instrumentation, vol. I-9, pp.
120132, September 1960.
[44] R. Beehler and D. Glaze, The Performance and Capability of
Cesium Beam Frequency Standards at the National Bureau of
REVIEW PAPERS
Standards, IEEE Transactions on Instrumentation and Measure-
ment, vol. 15, pp. 4855, March-June 1966.
[45] D. Glaze, Improvements in Atomic Cesium Beam Frequency
Standards at the National Bureau of Standards, IEEE Transac-
tions on Instrumentation and Measurement, vol. IM-19, pp.
156160, August 1970.
[46] D. Glaze, H. Hellwig, D. Allan, S. Jarvis, Jr., and A. Wainwright,
Accuracy Evaluation and Stability of the NBS Primary Frequency
Standards, IEEE Transactions on Instrumentation and Measure-
ment, vol. IM-23, pp. 489501, December 1974.
[47] D. Glaze, H. Hellwig, D. Allan, and S. Jarvis, Jr., NBS-4 and NBS-
6: The NBS Primary Frequency Standards, Metrologia, vol. 13,
pp. 1728, 1977.
[48] L. Lewis, F. Walls, and D. Howe, Prospects for Cesium Primary
Frequency Standards the National Bureau of Standards, from Pre-
cision Measurements and Fundamental Constants II, B. Taylor, W.
Phillips, eds., NBS Special Publication 617, pp. 2527, 1984.
[49] R. Drullinger, J. Lowe, D. Glaze, and J. Shirley, NIST-7, the New
US Primary Frequency Standard, Proceedings of the European
Time and Frequency Forum, pp. 549551, March 1993.
[50] R. Drullinger, J. Shirley, D. Glaze, and L. Hollberg, An Optically
Pumped Primary Frequency Standard, Proceedings of the 1988
Frequency Standards and Metrology Symposium, pp. 116119,
September 1988.
[51] J. Shirley, W. Lee, and R. Drullinger, Accuracy evaluation of the
primary frequency standard NIST-7, Metrologia, vol. 38, pp.
427458, October 2001.
[52] K. Gibble and S. Chu, Future Slow-atom Frequency Standards,
Metrologia, vol. 29, pp. 201212, 1992.
[53] M. Kasevich, E. Riis, S. Chu and R. DeVoe, rf Spectroscopy in an
Atomic Fountain, Physical Review Letters, vol. 63, pp. 612615,
August 1989.
[54] K. Gibble and S. Chu, Laser-Cooled Cs Frequency Standard and
a Measurement of the Frequency Shift due to Ultracold Collisions,
Physical Review Letters, vol. 70, pp. 17711774, March 1993.
[55] A. Clarion, P.Laurent, G. Santarelli, S. Ghezali, S. N. Lea, and M.
Bahoura, A cesium fountain frequency standard: Recent results,
IEEE Transactions on Instrumentation and Measurement, vol. 44,
pp. 128131, April 1995.
[56] H. Metcalf and P. van der Straten, Laser-Cooling and Trapping,
Springer-Verlag, New York, 1999.
[57] D. Wineland and H. Dehmelt, Proposed 10
14
< Laser Fluores-
cence Spectroscopy on T1
+
Mono-Ion Oscillator III, Bulletin of
the American Physical Society, vol. 20, p. 637, 1975.
[58] D. Wineland, R. Drullinger, and F. Walls, Radiation Pressure
Cooling of Bound Resonant Absorbers, Physical Review Letters,
vol. 40, pp. 16391642, June 1978.
[59] S. Jefferts, D. Meekhof, J. Shirley, and T. Parker, Preliminary
Accuracy Evaluation of a Cesium Fountain Primary Frequency
Standard at NIST, Proceedings of the 1999 IEEE Frequency
Control Symposium, pp. 1215, April 1999.
[60] W. Klipstein, C. Ekstrom, S. Rolston, and W. Phillips, Progress
Towards a Laser-Cooled Cesium Atomic Frequency Standard at
NIST, Gaithersburg, Proceedings of the 1998 IEEE Frequency
Control Symposium, pp. 912, May 1998.
[61] S. Jefferts, J. Shirley, T. Parker, T. Heavner, D. Meekhof, C. Nelson,
F. Levi, G. Costanzo, A. DeMarchi, R. Drullinger, L. Hollberg, W.
Lee and F. Walls, Accuracy evaluation of NIST-F1, Metrologia,
vol. 39, pp. 321336, 2002.
[62] T. Heavner, S. Jefferts, E. Donley, J. Shirley, and T. Parker, NIST-
F1: Recent improvements and accuracy evaluations, Metrologia,
vol. 42, pp. 411422, 2005.
[63] S. Weyers, U. Hbner, R. Schrder, C. Tamm and A. Bauch,
Uncertainty evaluation of the atomic caesium fountain CSF1 of
the PTB, Metrologia, vol. 38, pp. 343352, 2001.
[64] C. Vian, P. Rosenbusch, H. Marion, S. Bize, L. Cacciapuoti, S.
Zhang, M. Abgrall, D. Chambon, I. Maksimovic, P. Laurent, G.
Santarelli, A. Clarion, A. Luiten, M. Tobar, and C. Saloman, BNM-
SRYTE Fountains: Recent Results, IEEE Transactions on Instru-
mentation and Measurement, vol. 54, pp. 833836, April 2005.
[65] F. Levi, L. Lorini, D. Calonico and A. Godone, IEN-CsF1 Accu-
racy Evaluation and Two-Way Frequency Comparison, IEEE
Transactions on Ultrasonics, Ferroelectrics, and Frequency
Control, vol. 51, pp. 12161224, October 2004.
[66] N. Pavlis and M. Weiss, The relativistic redshift with 3 10
-17
uncertainty at NIST, Boulder, Colorado, USA, Metrologia, vol.
40, pp. 6673, 2003.
[67] E.Simon, P. Laurent, A. Clairon, Measurement of the Stark shift
of the Cs hyperfine splitting in an atomic fountain, Physical
Review A, vol. 57, pp. 436444, January 1998.
[68] S. Micalizio, A. Godone, D. Calonico, F. Levi, L. Lorini, Black-
body radiation shift of the
133
Cs hyperfine transition frequency,
Physical Review A, vol. 69, 053401, 2004.
[69] F. Levi, D. Calconico, L. Lorini, S. Micalizio, A. Godone, Mea-
surement of the blackbody radiation shift of the
133
Cs hyperfine
transition in an atomic fountain, Physical Review A, vol. 70,
033412, 2004.
[70] K. Beloy, U. Safronova, A. Derevianko, High accuracy calculation
of the Blackbody Radiation Shift in the
133
Cs Primary Frequency
Standard, Physical Review Letters, vol. 97, 040801, 2006.
[71] F. Levi, A. Godone, L. Lorini, Reduction of the Cold Collisions
Frequency Shift in a Multiple Velocity Fountain: A New Proposal,
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency
Control, vol. 48, pp. 847850, May 2001.
[72] F. Levi, A. Godone, L.Lorini, S.R. Jefferts, T. Heavner, C.Calosso,
The Multiple Velocity Fountain: A New Scheme for the Cold Col-
lision Shift Reduction, Proceedings of the 2001 Frequency Stan-
dards and Metrology Symposium, pp. 466468, September 2001.
[73] T. Parker, S. Jefferts, T. Heavner, E. Donley, Operation of the
NIST-F1 caesium fountain primary frequency standard with a
maser ensemble, including the impact of frequency transfer noise,
Metrologia, vol. 42, pp. 423430, 2005.
[74] M. Lombardi, NIST Time and Frequency Services, NIST Special
Publication 432, January 2002.
[75] E. Arias, The metrology of time, Philosophical Transactions of
the Royal Society A, vol. 363, pp. 22892305, July 2005.
[76] L. Hollberg, C. Oates, G. Wilpers, C. Hoyt, Z. Barber, S. Diddams,
W. Oskay, and J. Bergquist, Optical frequency / wavelength ref-
erences, Journal of Physics B, vol. 38, pp. 469495, April 2005.
[77] L. Ma, Z. Bi, A. Bartels, K. Kim, L. Robertsson, M. Zucco, R.
Windeler, G. Wilpers, C. Oates, L. Hollberg, and S. Diddams,
Frequency Uncertainty for Optically Referenced Femtosecond
Laser Frequency Combs, IEEE Journal of Quantum Electronics,
vol. 43, pp. 139146, February 2007.
NEW PRODUCTS
New Precision Measurements
AC Current Shunts
Measure-Techs
model El-9800
current shunt set
expands the AC
current measure-
ments to 100 A
and 100 kHz. The
conceptual design for the shunts was
developed by NIST and has been tested
and approved by the Air Force, where it
will be used in all AF Precision Measure-
ment Laboratories to certify current
levels up to these values.
For more information contact: Precision
Measurements Inc., 8025 Wheatland Ave.,
Unit A, Sunnyvale, CA 91352 USA or visit
www.measure-tech.com
New TEGAM PXI
Instrumentation
Amplifier
TEGAM, Inc. has
released its first PXI
(Peripheral Component
Interconnect (PCI)
eXtensions for Instru-
mentation) product, the Model 4040A,
50 MHz PXI Instrumentation Amplifier.
Designed to expand the measurement
range of PXI digitizers and analog inputs,
the Model 4040A offers 100V differential
inputs with 70 dB of common-mode
rejection ratio (CMRR). The bandwidth
ranges from DC-50 MHz with program-
mable gain, attenuation, offset and filters.
The Model 4040A comes complete with
VISA compliant drivers for LabVIEW8,
Microsoft C++ and Visual Basic. In addi-
tion, an interactive front panel VI pro-
vides manual control of all of the boards
features. Other specifications include:
1 M and 50 input impedance; Pro-
grammable Attenuation/Gain/ 100,
10, x1, x10, x100; and 50 output.
For more information, visit
www.tegam.com/4040a
Updated AssetSmart Software
AssetSmart has released the latest
version of its calibration management
system. SMART/CMS 3.6.0 contains
over 100 improvements and enhance-
ments to further enable calibration
organizations to standardize business
processes and integrate relevant data into
a single data repository. The newest
release includes enhanced shipping and
receiving capabilities and contains new
features and functionality for the legal
metrology market. SMART/CMS 3.6.0
also supports adoption of new technolo-
gies such as RFID and mobile device
integration, and has been enhanced to
meet or exceed industry requirements
including UID, ANSI/NCSL Z540, ISO
9000/9001, ISO/IEC/EN 17025 and
FDA 21CFR Part 11.
For more information, contact AssetSmart
at sales@assetsmart.com or (310) 450-
2566 or visit www.AssetSmart.com
Transcat Adds
New Vision
System to Primary
Dimensional Lab
Transcat has added the
Starrett Galileo Vision
System to its Primary
Dimensional Laboratory.
The Galileo is a high-accuracy multi-
sensor inspection system incorporating
both vision and touch probes and can
perform both manual and automated part
calibrations. Applications range from
simple measurements like length, width,
and diameters, up to complex 3D custom
parts, following GD&T practices. The
key benefit of multi-sensor systems is their
ability to measure complex dimensional
forms and surface relationships that may
be impossible for a single-sensor systemto
capture. With this vision system, Transcat
can perform accurate calibrations of
custom gages and fixtures for customers
in highly regulated industries, such as for
medical devices manufacturers.
For more information, visit our website
www.transcat.com
Modus Metrology Offering
Torque & Alignment/
Measuring Services
Modus Metrology Services, a division of
Omega Energy Services of Oak Ridge
TN, is offering Laser Tracker alignment
and measuring services across the South-
eastern States beginning November 2007
using a state-of-the-art Leica LTD840
Laser Track-
ers. In addi-
tion, they are
offering torque
wrench cali-
bration service
over multiple
ranges up to
2500 ft/lbs. They are also beginning con-
struction of a new 18,000 square foot
facility, which will house a Primary level
metrology lab offering calibration serv-
ices in Physical, Electrical and Dimen-
sional disciplines. This facility will
include the Leitz Infinity Coordinate
Measuring Machine. With this machine,
Modus Metrology will also have the
capability to perform classified and pro-
prietary calibrations.
For more information contact Ed Pritchard,
Business Manager, at (865) 483-9555 or
epritchard@modusmet.com
DynaCal and Measurement
Assurance Tech. Merge
DynaCal has announced its merger with
Measurement Assurance Technology
(MAT) of Dallas, Texas. MAT has over
15 years experience in the calibration
service business and also Buys/Sells,
Rents/Lease High-end T&M equipment
world wide. This merger will provide
both cali-
bration
support
and asset
acquisi-
tion/dispo-
sition to
customers. Joining forces with MAT, has
elevated our capacity to serve our cus-
tomer to another level, not found any-
where else. The ability to deliver asset
management support, has been received
exceptionally well. said Felipe Narcio,
Dynacal. Our goal is to provide the
highest level of customer support.
Narcios premise of how to best serve the
customer was the main reason for select-
ing MAT as its partner. DynaCal chose
MAT because of their well-established
asset management support team and
excellent customer service.
For more information call 1-866-DYNACAL
or visit www.dyna-cal.com
More on page 92
NEW PRODUCTS
NEW PRODUCTS
FREE Pressure Proficiency
Testing
Crystal Engineering is offering FREE pro-
ficiency testing services for interlabora-
tory comparisons in pressure using their
digital pressure gauges, M1 and XP2i,
which have high repeatability, stability
and ruggedness. The XP2i gauges cover
the range 15 psi to 10,000 psi and have
an accuracy of 0.1% of reading, down to
20% of the range. The M1 gauges cover
the range 30 psi to 10,000 psi and typi-
cally have an accuracy of 0.2% of
reading, down to 20% of the range.
Using these gauges, laboratories can eval-
uate their performance when calibrating
the same or similar pressure items.
For more information about this free
service, contact Crystal Engineering at
800-444-1850,
sales@crystalengineering.net or
www.crystalengineering.net
IsoTech Introduces the
MicrosKanner
The new MicrosKanner from IsoTech is
an expansion unit that adds 10 channels
to a MicroK preci-
sion thermometry
bridge. As many as
10 MicrosKanners
can be added to the
MicroK bridge, expanding its capacity
from 3 channels to 92 without affecting
measurement accuracy. The MicroK has
an uncertainty of only 0.4 ppm for the
MicroK-400 and 0.8 ppm for the
MicroK-800. The MicrosKanner has
zero drift for resistance measurements
and only 3 ppm/year drift for voltage
measurements. The MicrosKanner can
be used for PRT, thermocouple and
thermistor measurements.
For more information call 802-863-8050 or
visit www.isotechna.com
ACLASS acquired by
ANSI-ASQ National
Accreditation Board
ACLASS Accreditation Services has been
acquired by the ANSI-ASQ National
Accreditation Board. ACLASS is nowpart
of the American National Standards Insti-
tute (ANSI) and ASQ family. ACLASS
will continue to offer ISO 17025 accredi-
tation to calibration and testing laborato-
ries as well as 17020 accreditation for
Inspection Bodies and Guide 34 for Refer-
ence Material Producers. ACLASS is a
signatory to the ILAC, APLAC and IAAC
multilateral recognition arrangements for
ISO/IEC 17025. This acquisition allows
ACLASS to strengthen its commitment to
providing technically competent accredi-
tation services. ACLASS customers will
experience no change with their existing
accreditation. This acquisition will not
change the current management structure
of ACLASS, our processes, policies and
procedures.
For more information contact: Roger Muse
703-351-9139 x202 or visit
www.aclasscorp.com
! CALIBRATION FROM 100 MICROHMS
TO 10 TERAOHMS
! DETERMINATION OF RESISTOR
TEMPERATURE COEFFICIENTS
ALPHA AND BETA
! STATE-OF-THE-ART LAB
! SPECIAL TESTING
! EXCELLENT UNCERTAINTIES
! RAPID TURNAROUND
RANGE BEST UNCERTAINTY
0.0001 1.5 /
0.001 1.1 /
0.01 0.7 /
0.1 0.4 /
1 k 0.28 /
10 k 0.15 /
100 k 0.45 /
1 M 0.56 /
10 M 3.8 /
100 M 5 /
1 G 10 /
10 G 20 /
100 G 50 /
1 T 100 /
10 T 350 /
Resistance Calibration
Process Instruments, Inc. 615 E. Carson Street Pittsburgh, PA 15203 Phone 412-431-4600 Fax 412-431-3792
www.procinst.com
FOR THE SCOPE OF ACCREDITATION
UNDER A2LA CERTIFICATE 1894.01
! OTHER LOW UNCERTAINTY ACCREDITED CALIBRATIONS
INFARED CALIBRATION
PRESSURE CALIBRATION
TEMPERATURE CALIBRATION SUPPORTED BY
FIXED POINTS ON THE ITS-90
ROCESS
PI
I n c.
NSTRUMENTS
Product information is provided
as a reader service and does
not constitute endorsement by
NCSLI. Contact information is
provided for each product so
that readers may make direct
inquiries.
Veriteq Instruments ofers a complete solution for Cal Lab
ambient condition monitoring applications with:
Superior 1-year temperature and relative humidity
measurement accuracy
Fast, easy, scalable deployment
Continuous monitoring and 24/7 alarm notifcations of
out-of-spec conditions
Monitoring of multiple locations fromany PC on the network
Contact us for more information or to discover why so many
companies are replacing their chart recorders with a superior
Veriteq solution.
V E R I T E Q I N S T R U M E N T S , I N C .
1 1 0 - 1 3 7 9 9 C o mme r c e P a r k wa y
R i c h mo n d , B C , C a n a d a , V 6 V 2 N 9
Relying on chart recorders just isnt reliable!
1 8 0 0 6 8 3 8 3 7 4 ( N o r t h A me r i c a )
Te l 6 0 4 2 7 3 6 8 5 0
w w w. v e r i t e q . c o m
ADVERTISER INDEX
Agilent Technologies www.agilent.com .................................... 39
The American Association for Laboratory Accreditation
(A2LA) www.a2la.org .......................................................... 21
CPEM 2008 www.icpem.org/2008 ............................................ 90
DH Instruments, Inc. www.dhinstruments.com ...................... 93
Dynamic Technology, Inc. www.dynamictechnology.com .... 7
Essco Calibration www.esscolab.com .................................... 94
Fluke / Hart Scientific Corporation
www.fluke.com .............................. Inside Front Cover, 12, 64
Gulf Calibration Systems
www.gcscalibration.com ........................ Outside Back Cover
Heusser Neweigh www.neweigh.com .................................... 95
Integrated Sciences Group www.isgmax.com ...................... 4
INSCO www.insco.us ................................................................ 65
Isotech www.isotech.com ........................................................ 59
Masy Systems, Inc. www.masy.com ........................................ 52
Mensor Corporation www.mensor.com .................................. 13
Morehouse Instrument Co. www.mhforce.com ...................... 53
NCSLI Conference & Symposium www.ncsli.org/conference/ 15
NCSLI Training Center www.ncsli.org .................................... 96
Northrop Grumman Corp. www.northropgrumman.com ...... 8
PPM Instruments www.ppminc.com ...................................... 96
Process Instruments Inc. www.procinst.com ........................ 92
Rotronic www.XXXXXXXXXXX.com .......................................... 65
Simmons Memorial Scholarship
www.simmons-scholarship.com ............................................ 20
TAC/Tour Andover Controls www.tac.com/pe ...................... 94
Thunder Scientific Corporation
www.thunderscientific.com . . . . . . . . . . . . . Inside Back Cover
Vaisala Inc. www.vaisala.com .................................................. 90
Veriteq www.veriteq.com .......................................................... 95
The new 9140 Series
Field Metrology Wells
deliver maximum
portability, speed,
and accuracy at a
minimum price
The 9140 family from Flukes
Hart Scientic division light-
ens your eld calibration
workload at every turn. Their
compact size and weight
make them the most portable
eld calibrators in their class.
Fast heating and cooling
takes you through your work-
load at twice the speed of
competitive dry wells. And
automated and documented
calibrations save even more
time. Plus the built-in refer-
ence thermometer, two mea-
surement channels, and 24 V
loop power supply ensure top
accuracy without extra tools.
Fluke. Keeping your world
up and running.
To learn more about why

9140 Series Field Metrology Wells
are the best in the eld go to
www.hartscientic.com, or call
1-800-GET-HART (438-4278).
Best in Field.
2007 Fluke Corporation.
All rights reserved. Ad 02133
C l OK_____
L
ICS# 070887 Fluke
9
measure
NCSL INTERNATIONAL
D
e
c
e
m
b
e
r
2
0
0
7
N
C
S
L
I
n
t
e
r
n
a
t
i
o
n
a
l
m
e
a
s
u
r
e
T
h
e
J
o
u
r
n
a
l
o
f
M
e
a
s
u
r
e
m
e
n
t
S
c
i
e
n
c
e
V
o
l
.
2
N
o
.
4
In This Issue:
Power Measurements
Electromagentic Metrology
Challenges in the U.S. DOD

NCSLI Measure 2007 Dec

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

NCSLI Measure 2007 Dec

Uploaded by

Copyright:

Available Formats

measure

To learn more about why

for i ts l eadi ng cal i brati on software products that hel p you

provides high technology with a human touch.

For more information, send email to

200mg 0.008mg 0.008mg

is the conventional mass difference between the SEN-

Table 6. Degrees of equivalence and levels of measurement

200 mg 0.020 mg 0.020 mg

Humidity goes Digital

To learn more about why

You might also like