31 Assessment of Health
Outcomes
KEY POINTS
Any single health outcome can give only a partial view of the
impact of a disease on a patient.
Core sets are minimal, but not exclusive, domains of
outcomes agreed on by professional groups as important to
include in studies; they are available for several rheumato-
logic conditions.
Defining measurement need is key to the choice of the right
instrument.
Choosing an instrument follows a step-by-step
process—looking for evidence of practical aspects
of using the instruments and statistical properties.
If an instrument lacks evidence of a certain property, a study
can be conducted to create the evidence, rather than
abandon the instrument.
In an era of rising health care costs, increased choice, and
greater provider accountability,1 health outcome ­ measures
have become essential tools for researchers, ­clinicians, and
funding bodies. By definition, outcomes refer to “all possible
effects of a disease or intervention.”2 Outcomes cover a spec-
trum of the burden of arthritis on a patient from biomark-
ers of disease to subjective appraisals of overall well-being.
Other chapters refer to some of the most common measures
of health and disease encountered in rheumatology, includ-
ing the Disease Activity Scale (DAS, DAS28),3 the Health
Assessment Questionnaire (HAQ),4 and the SF-36 (short
form, 36 items).5 Using any one of these health outcome
assessments is like looking out a window in a house, with
the burden of arthritis the landscape outside. Each window
in the house provides a view of the outside world, but it is a
specific view defined by the size of the window and the side
of the house it is on. Another window may offer a slightly
better angle on what you would like to see. Different health
outcome assessments can have a degree of overlap in their
views, in which case an informed choice needs to be made
between them, whereas others can hold quite distinct views.
One assessment might be useful to compare the burden of
arthritis against the general population, another might be
useful to assess differences in the prevalence in different
subgroups, and yet another might be useful to measure the
specific benefits of an arthritis intervention.
This chapter focuses on describing the different windows
clinicians have on the burden of arthritis and how they
relate to each other. A framework is provided for ensuring
that a selected measure is the right one for a given need. This
chapter addresses four questions: (1) What health outcome
assessment tools are available generally and ­specifically for
Dorcas Eleanor Beaton  •  Maarten
Boers  •  Peter Tugwell
use in rheumatology? (2) How do the different assessment
tools relate to one another? (3) How does one characterize
what one needs to measure? (4) How does one find a mea-
sure that can meet that need?
WHAT HEALTH OUTCOME ASSESSMENT
TOOLS ARE AVAILABLE?
In reading the rheumatology literature, and in monitor-
ing the clinical care of patients, certain highly relevant
outcomes emerge. A group of these often emerge and are
termed core sets of outcomes.
DISEASE-SPECIFIC MEASURES—THE CORE SETS
Core sets are the minimal, but not exclusive, set of domains
to be measured in a study of arthritis. Historically, they fol-
low the Ds of outcome measurement in arthritis: disability,
disease activity, damage, discomfort, dissatisfaction, and
death.6,7 They are usually recommended by groups such as
OMERACT, EULAR, ILAR, or ACR or groups formed
around specific diseases, such as ASAS for ankylosing spon-
dylitis or GRAPPA for psoriatic arthritis. All core sets have
a great interest in agreeing on a common set of relevant and
psychometrically sound outcomes that would allow them to
compare findings across studies and of clinical care.
Table 31-1 presents the list of core sets for clinical trials
in six types of arthritis7-16; it also shows what each group rec-
ommends as additional domains or as needing more research
before they become core set members. The first column on the
left of Table 31-1 is the core set for longitudinal observational
studies in rheumatology by Wolfe and colleagues7 with some
minor additions. Wolfe’s core set is broader than the core sets
presented in other columns because observational studies are
often looking for a broader range of outcomes that are rel-
evant for studies outside of treatment trials. It also is designed
to be used across different forms of arthritis. The table also
uses this broader list of outcomes as an axis against which the
other core sets can be described. The remaining columns show
that across different types of arthritis, the core sets have many
common elements; most contain or recommend pain, physi-
cal function, patient and clinician global assessments, and
markers of inflammation. Many also include disease activity
indices, which are often an aggregation across other clini-
cal findings (e.g., joint count, acute-phase reactants, global
ratings of severity) into a score reflecting the activity of the
disease at that point in time. Some core sets contain domains
reflecting the unique aspects of the disease (e.g., spinal mobil-
ity in ankylosing spondylitis)17 or the unique target of the
study using the outcomes (fractures are core outcomes only in
osteoporosis studies focused on fracture prevention).18
463
 464

Table 31-1  Core Sets for Six Rheumatologic Conditions and for Longitudinal Observational Studies*
BEAToN 

Clinical Trial Core Sets of Domains by Disease Group

| 

Longitudinal Study Core Rheumatoid Systemic Lupus Ankylosing

Set of Domains7 Arthritis15,16 Osteoporosis11 Osteoarthritis13 Erythematosus14,19 Spondylitis8 Psoriatic Arthritis10,12
Health status/quality of life ✓
Quality of life R (utility) R (BL)(†) R ✓ ✓
Symptoms ✓ Pain R (back)(†) ✓ Pain R (fatigue) ✓Pain ✓ Pain
✓ Fatigue
Physical function ✓ Disability R(†) ✓ R ✓ ✓
Psychosocial R R

Disease process ✓
Aggregate index ✓ (EULAR DAS) ✓ DAI, R = severity Pending ✓ DAS
Biomarkers ✓ Biochemical O
Joint tenderness ✓ 28 or 68 joints ✓ Peripheral (44 joints) ✓
Assessment of Health Outcomes

Enthesitis Enthesitis‡
Joint swelling ✓
Joint stiffness O ✓ Spinal stiffness
Global ✓ Spinal mobility
Patient ✓ ✓ ✓ ✓
Physician ✓ R ✓
Acute-phase reactants ✓ O R ✓‡ ✓
Damage ✓
Radiography or imaging ✓ >1 yr ✓ Bone mineral density ✓ >1 yr ✓ Spine and hip§ ✓ Structural
Deformity
Surgery
Organ damage R (BL)/ ✓ (†) fractures ✓ Damage index ✓Skin disease
R (BL)/ ✓ (†) Change height
Disadvantages
Toxicity effects ✓ R ✓ ✓
Death ✓

Dollar costs [R] R R(†) R ✓

Work disability [R] R
*The left-hand column is the broader set of measures recommended by Wolfe and colleagues7 for longitudinal studies. It serves here as a broader range of outcomes, and an axis for the organization of core outcomes
in the other conditions (columns).
✓, core domain; R, recommended for further research and possible inclusion in core set; O, optional outcome (osteoarthritis category only).
Osteoporosis: Core set depends on focus: BL, bone loss studies; †, studies aiming to reduce fracture rates.
Ankylosing spondylitis: Core set elements vary depending on focus of study: ‡, clinical records and symptom modifying; §, disease modifying only; others = all.
DAI, disease activity index; EULAR DAS, European League Against Rheumatism Disease Activity Scale (revised = DAS-28, 28-joint count).
 PART 4  |  BROAD ISSUES IN THE APPROACH TO RHEUMATIC DISEASES
Table 31-1 focuses on the core domains that should be
measured; the next step is deciding on the instrument that
is able to provide that well in a reproducible, accurate man-
ner. In some cases, an instrument choice has been suggested
(e.g., the HAQ for disability in rheumatoid arthritis). Other
times several options are provided. Strand and cowork-
ers19 reviewed six disease activity indices in systemic lupus
erythematosus and found they gave comparable results. In
some cases, the domains are shared, but the measurement
technique varies within or by disease; in rheumatoid arthri-
tis, the DAS28 uses 28 joints,3 and in ankylosing spondy-
litis, 44 joints are counted.17 Some of the more commonly
encountered instruments in arthritis are briefly reviewed.
Health Status/Quality of Life
General Health Status. Generic health outcomes provide
information on an aspect of health across many conditions
so that theoretically comparisons can be made to compare
the burden of low back pain with that of arthritis or diabe-
tes. This comparison depends on the ability of that measure
to capture the burden in a disease group well. Generic mea-
sures have advantages of allowing comparisons across dis-
eases and covering a broader range of health issues—things
that may have been overlooked in a core set (e.g., mental
health issues). Often because of their breadth, however, the
generic measures tend not to delve as well into the depth of
experience in any one disease. Arthritis-related fatigue is not
detected well in many generic measures because a generic
measure asks about being tired or not sleeping well. Because
of this, a generic measure is usually weaker in its ability to
detect specific changes and their sensitivity to different lev-
els of disease activity and should usually be supplemented
with disease-specific measures (described previously).20
Two commonly used generic measures are the Sickness
Impact Profile (SIP)21 and the SF-36.22 The SIP is a 136-
item list of illness behaviors that provides a weighted score
for the impact of a disease across 12 categories, such as bodily
pain, work and role functioning, and dressing,21 which lead
to global scores (physical, psychosocial, and overall). The
SIP has been shown to measure illness across a wide vari-
ety of health conditions.20 The SF-36 is a 36-item question-
naire of which 35 items are used to obtain eight domain
scores (including physical functioning, mental health, role
functioning, and pain) scored on a 0-to-100 scale (with
100 = better health)22 and two summary scores (mental and
physical); the questionnaire is scored with normal of 50 and
standard deviation of 10. The SF-36 and briefer SF-12 are
well supported on the website (www.qualitymetric.com) and
through manuals that supply age and disease group distri-
butions of scores.23 Direct comparisons of generic measures
have shown differences in scores and health states attribut-
able to the choice of measure.24-26 Studies or clinical results
may not be comparable to each other if they are using differ-
ent health status scales.
Utilities—Value of Health State. Utility scales offer an
overall score for the value of a health state, setting death
at 0 and full health at 1. The emphasis is not on describing
the state, but on assigning a value, worth, or preference to
that state.27,28 Utilities are needed for economic appraisals
and form the health assessment for cost per quality-adjusted
465
life-years estimations. Utility states can be obtained by di-
rect or indirect methods. Direct methods, such as standard
gamble and time tradeoff, involve the respondent work-
ing through exercises to elicit the value for his or her own
health state against things such as time or more or less fa-
vorable health situations.27 Indirect methods capture the
state with standardized questions and apply predetermined
weights.28 Examples include the EQ-5D which is five items
(three response categories) combined to describe a health
state. Similarly, the Health Utility Index gathers informa-
tion on six or seven dimensions of health (depending on the
version) on five-item to six-item response scales to define a
health state.28 Both these scales then use weights determined
in different populations to assign the value to these health
states—hence the “indirect” weighting. The absolute value
obtained across these different approaches varies.27
Generic measures of health and utility scores are very
broad. They often do not perform as well as the more specific
measures described in the following sections because they are
designed to allow comparisons across populations and need to
include items that might not be relevant in arthritis. In some
areas, there are measures of quality of life that are designed
for that disease, such as rheumatoid arthritis or osteoporosis,
which offer a measure of the broad concept of quality of life
in a disease-specific manner. Such measures would not allow
comparisons across diseases. At the time of this writing, these
were not yet recommended as core instruments.
Symptoms. Pain is usually measured using a 10-cm visual
analog scale or a 0-to-10 numeric rating scale of the intensity
of the symptoms.29 This simple measure has been well tested
and is easily understood by patients. Fatigue is another im-
portant symptom and quite distinct from being “tired.” The
ankylosing spondylitis modified core set contains fatigue,
and it is a recommended area of further research in rheuma-
toid arthritis and lupus. Measures are being tested or devel-
oped at present. Ankylosing spondylitis is currently using the
10-cm visual analog scale of fatigue from the Bath Ankylos-
ing Spondylitis Disease Activity Index (BASDAI).30
Disability Scales. Physical disability in rheumatoid arthritis
and osteoarthritis is often measured using the Health Assess-
ment Questionnaire–Disability Index (HAQ-DI),31 which
covers 20 items looking at different aspects of daily func-
tioning. Patients score each item on a 0-to-3 scale, where
3 represents the greatest disability. Scores are obtained for
each domain and summed into a total score expressed on
the same 0-to-3 scale. Scores are adjusted to a 2/3 if an aid
is used to ­complete a task. More details on the HAQ-DI are
widely available in print and on the Internet.
There are other scales asking about physical function such
as the Arthritis Impact Measurement Scale (AIMS)32 and the
AIMS233 and measures with even more specific foci, such as the
Western Ontario McMaster (WOMAC) osteoarthritis index,
which is commonly used in hip and knee osteoarthritis,34 and
the AUSCAN osteoarthritis index for hand osteoarthritis.35
Disease Process (Activity, Severity)
Core sets often include indices of disease process. Disease
process can be divided into activity (inflammatory activity)
and severity (overall severity of disease). There are several
 466 BEAToN  |  Assessment of Health Outcomes
disease activity indices, the most commonly known being
the DAS36 and DAS283 in rheumatoid arthritis. Using a
subset of the core outcomes (i.e., acute-phase reactants,
joint counts, global ratings), the EULAR group formulated
a weighted index that provides a score of 2 to 10 (DAS)
or 0 to 9 (DAS28). From this, cutoffs were established to
define high, moderate, and low disease states. The low dis-
ease state (DAS28 <2.6) is considered an indicator of remis-
sion of arthritis and is touched on again later. More recently,
the concept of “stable remission” has been proposed; it is
defined as an initial DAS28 of less than 3.2 and a lack of
change in DAS over time of 1.2 (2 standard errors).37 Dis-
ease activity indices track the level of inflammatory activity.
There are others, such as the BASDAI30 or the six available
in systemic lupus erythematosus.19 When more than one is
available, look for direct comparisons such as Strand’s to see
if similar information is provided.19,38
Damage Indices
A great deal of work has gone into measures of joint damage
in arthritis. van der Heijde and Landewe17 provide a succinct
summary of the Sharp, van der Heijde, and Larsen/Scott tech-
niques. Guidelines should be followed closely, focusing on the
hands and feet. Changes in radiographic progression of joint
damage often are measured by the smallest detectable differ-
ence, the boundary between measurement error (day-to-day
variability) and change discernible from error.39,40
Disadvantages—Toxicity/Adverse Events
Medical and nonmedical management of many rheumatic
conditions carries a risk of toxicity and adverse events,41
many of them unexpected. A comprehensive documenta-
tion of a range of adverse events is important in outcome
assessments separate from the treatment benefits.42 An
OMERACT group is currently working on standardizing
reporting of toxicities in rheumatologic trials.43
Death
Arthritis is associated with increased mortality, and arthritis-
specific mortality should be monitored. Death is not specifically
mentioned in the disease-specific core sets for clinical trials,
but would be important to monitor in observational studies.
Attributing death to arthritis is challenging given its depen-
dence on documentation at time of occurrence, which may or
may not be linked to underlying arthritis in the coding.
Dollar Costs
The economic burden of arthritis and the cost of care are core
outcomes for psoratic arthritis and recommended for consid-
eration in observational studies and rheumatoid arthritis,
osteoporosis, and lupus disease groups. Standards for what
should be included in a cost analysis are under development.
Disease Self-Management
Lorig’s work in self-management programs often have
focused on improving self-efficacy, which has been found to
reduce pain and health care use.25 Self-efficacy is a health
outcome. Similarly, Tugwell’s “effective consumer” captures
the degree to which the patient is effectively managing
his or her own health care decisions, interactions with the
health care team, and disease monitoring.44 This may be a
reasonable target for self-help interventions.
EMERGING DOMAINS
Work Productivity and Disability
With the shift toward more aggressive management of ear-
lier disease, more people with arthritis are working, and
outcomes need to shift to track work disability (absentee-
ism and at-work productivity loss).7 Work is hard to mea-
sure because it depends on the job and the organization the
individual works in. Absenteeism can mean many things,
and measures should articulate how this was operational-
ized—full days off work, days on insurance payments, or full
and part days off work. More challenging still is measuring
the difficulty someone is having at work (presenteeism).
There are several measures—16 were found in a more recent
review45—but there are few direct comparisons of these con-
ceptually diverse instruments. The most commonly used is
the work limitations questionnaire (amount of time expe-
riencing difficulty).46 Two scales developed in arthritis are
promising—Gignac’s Work Activity Limitations Scale
(amount of difficulty experienced)47 and Gillworth’s Work
Instability Scale (risk of future work loss).48 Direct compari-
sons of these measures are under way.
Nonpaid Work
Participation in valued nonpaid roles, such as parenting,
volunteer work, or leisure activities, can be an impor-
tant aspect of the burden of disease.49 Outcome measures
reflecting this are needed to capture fully the concept of
participation.
Patient-Specific Domains
Patient-specific scales, including the MACTAR or PET in
arthritis,50,51 allow the patient to nominate his or her own
scale content, within a guided framework. Most patients
report three to five items that are particularly salient to them.
A surprising number of these scales have been developed.52
Each taps relevant content for that patient, and because of
this they also are responsive to change.53 The challenge is
in the mathematics and how to analyze the numeric score
when the items vary across patients. Analysis that focuses
on individual level quantification is likely best.
Satisfaction with Health Outcomes
Satisfaction scales are often linked with the goals of a health
care organization and focus on the attributes of the structure
and process of care. Instruments developed to look at satis-
faction with a specific health end point (e.g., how satisfied
are you with the results of your surgery)54 become a health
outcome. Satisfaction with outcome is complex, and Hudak
and colleagues55 point out the complex balance of experi-
ences and ability to “live with” ongoing limitations that
influence a patient’s response.
 PART 4  |  BROAD ISSUES IN THE APPROACH TO RHEUMATIC DISEASES
Sleep
The OMERACT patient group has identified sleep qual-
ity as an important domain for outcome assessment.
The concept and measurement of it are expected to be
addressed at the upcoming OMERACT 9 meeting in
2008.
HOW DO OUTCOME MEASURES RELATE
TO ONE ANOTHER?
With an array of potential measures, or windows to view
the burden of disease, how does one organize them to
get an accurate picture of the whole? Conceptual frame-
works help one to understand how domains, such as those
described previously, theoretically relate to one another
when applied in research or clinical practice. They also
usually provide an operational definition of each of
their domains, which becomes essential when choos-
ing between instruments or deciding how to model the
outcome of an intervention and its modifying factors.56,57
Conceptual frameworks facilitate accurate communica-
tion and the generation of hypotheses of understanding
of a disease process and impact. In the past, the most
common conceptual framework was the main (sometimes
causal) pathway of a biomedical model: Pathology leads
to organ/system changes leads to pain/symptoms leads to
functional loss leads to diminished quality of life. This is
similar to a Wilson and Cleary model58 and might help
in understanding that function and quality of life might
relate more closely than sedimentation rate and quality of
life—based on their proximity and distance in the model.
Expanding on this type of model are conceptual frame-
works such as Nagi59 or Verbrugge’s60 disablement process.
In these models, there is a main pathway, with slightly
different definitions, but there also are boxes of influence
from outside this pathway—the patient’s personal factors
and environmental factors. In 2001, the World Health
Organization ratified the International Classification of
Functioning (ICF),61 which offers an even more expanded
framework based on a biopsychosocial understanding of
disease. It is receiving wide endorsement in arthritis.56,62
In this model (Fig. 31-1), there are three main elements of
burden. A disease can affect an individual by impairment
(body part), activity limitations (individual’s ability to
do a task), or participation restriction (restrictions in the
execution of the individual’s roles in life situations). The
boxes in Figure 31-1 are joined by bidirectional arrows
because this model suggests that the right-side boxes also
could influence the left with secondary problems (e.g.,
joint contractures or wounds secondary to prolonged bed
rest). For the ICF, disablement is not a linear progression;
it is a dynamic interaction influenced by several factors.57
The ICF strongly emphasizes the influence of personal
factors and environmental factors on all three domains.
Several initiatives are under way to examine the fit and
usefulness of the ICF framework in various forms of arthri-
tis.57,63,64 Readers are referred to Jette and Keysor65 for an
excellent comparison of the ICF and Verbrugge models in
the context of arthritis.
In addition to providing the conceptual framework, the
ICF undertook the task of a classification system listing
467
Health condition
(disorder/disease)
Body function
and structure
Activities Participation
Environmental Personal
factors factors
Figure 31-1  The International Classification of Functioning conceptual
framework showing the hypothesized relationships between domains of
impairment, activity limitations, and participation restrictions.
all possible categories of impact falling under each of the
five main headings. Many groups, particularly in arthritis,
have reviewed the categories and developed a list of the
categories relevant to that form of arthritis. This list sug-
gests which of these categories should be reflected in core
set measures in arthritis, offering a form of standard for the
content of scales.63
Frameworks define the realm of outcomes that should
be considered, and the hypothetical relationships between
them. They form the basis for understanding observa-
tions, testing hypotheses, or planning and executing an
analysis. Shifting frameworks is challenging because dif-
ferent tools may vary in how they define certain aspects
of health or disability. Shifts can prompt fruitful rethink-
ing of concepts, however, and how they relate to each
other.64
HOW DOES ONE CHARACTERIZE WHAT
ONE NEEDS TO MEASURE?
Just as investigators define a research question before
embarking on a clinical trial, so should users of health out-
comes define a measurement question before choosing an
instrument.
WHY MEASURE?
Clarity about a measure’s intended purpose helps ensure the
right one is selected. Measures can be used in three ways: to
describe people at one point in time, to predict a future
state, and to measure change over time.66,67 This chapter
focuses on the purposes used for health outcome assessment:
describing an end point state in a trial, which is descriptive,
and evaluating change over time.
WHAT TO MEASURE
An understanding of concepts and definitions is important.
It is not good enough to decide to measure physical func-
tion, for example; a better outcome would be physical func-
tion at the level of disability according to Verbrugge and
Jette.60 Similarly, when measuring pain, is intensity more
important than frequency? What about the degree to which
 468 BEAToN  |  Assessment of Health Outcomes

pain interferes with daily activities? Questions such as these
should be addressed before any instruments or core sets are
reviewed. The instrument should meet the need, and not
the reverse.
WHO CONSTITUTES THE TARGET POPULATION
The target population is crucial, but often overlooked. A
given instrument may work well in severe osteoarthritis of
the hip, but not be sensitive to the early symptoms of the
disease. Equally important is to consider if one wants to
measure for an individual patient or for describing a group of
patients as a whole. The former demands much higher lev-
els of measurement properties (e.g., reliability coefficients
>0.90 as opposed to 0.75 to 0.80 being adequate for group
descriptions).66
SELECTING THE OUTCOME THAT CAN
MEET THE MEASUREMENT NEED
The selection of an outcome measure depends entirely
on a clear understanding of measurement need. Often a
well-used instrument is used, rather than looking for one
that matches the concept, population, and purpose. Many
guidelines that offer more detail than can be described
here are available, particularly for the acceptable levels
of reliability and validity.66-72 This section describes a
decision-making process for fit between a given instru-
ment and the clinician’s need (Fig. 31-2). This process
builds on the work of Law72 and the OMERACT filter69
and highlights key understandings in each area from the
published guidelines.
This decision-making process emphasizes three things.
First, it begins by stating the measurement need (why,
what, and in whom), which reinforces that a candidate
measure meet one need, but not the next. Second, it
emphasizes that a lot of the appraisal can be done with-
out statistics. It is done by appraising the questionnaire
or instrument itself and knowledge of its administration.
Third, the inability to affirm each stage suggests that
there is no need to continue. At the later data-based
stages, the clinician may choose to run a small study to
create the evidence (the “do-it” loops) in patients, rather
than abandoning the instrument that seems like a good
candidate. Given these three key features, the process
from left to right is reviewed next.

Need: Concept Population Intended purpose: describe evaluate change

Candidate measure:

1. Matches
target concept? Boxes marked with “do it loop” are
those where you can create the
No evidence and continue on
2. Feasable Blue boxes = pre-data evaluation
to use? Yellow boxes = data-based evaluation
No

a. One point in time (descriptive)

3. Does it measure what Reliability
it says it does? (truth) -internal consistency
No
-inter-rater reliability
Face Construct validity
Content Construct
-differentiates high/low levels
-acts as expected with other indicator

4. Does it have b. Change over time (evaluative)

purpose-specific -test-retest reliability
properties in your -inter-rater reliability
No -responsive to change that is
population?
similar to target situation

Benchmarking
Choose scores (states)
another 5. Is it interpretable?
tool No Change
thresholds

Combinations:
change and state
Good fit for your needs!

Figure 31-2  Algorithm showing decision-making process for the fit of a candidate measure with your measurement need. The left-hand side of the
page is done by appraising the instrument and its instructions. The right-hand side requires numeric evidence of the relevant measurement properties.
Many instruments are weeded out as a poor fit in steps 1, 2, and 3. The “do-end” loop denotes stages at which you can pause to create evidence if it is
missing and not have to abandon the instrument.
 PART 4  |  BROAD ISSUES IN THE APPROACH TO RHEUMATIC DISEASES
STEP 1: IS THERE A MATCH BETWEEN
THE INSTRUMENT’S CONCEPT AND THE
MEASUREMENT NEED (CONCEPT,
POPULATION, PURPOSE)?
An operational definition of the target concept, the appli-
cable populations (patients or general population), and
intended purpose should be articulated by the developer and
match your current need.42,68,71,73 If this is not the case, or if
it is not a good match, start with another candidate measure
because this one would not work.68
STEP 2: IS IT FEASIBLE TO USE?
Feasibility covers the practical aspects of using this scale in
the intended setting.42,69,72,73 Does it take too much time? Are
the licensing costs too high? Does it require special equip-
ment? Is it too burdensome for your patients (language,
literacy, acceptability of questions)? Is it formatted well on
the page, and do the responses make sense given the tar-
get and the question? Are the questions phrased in a clear
and simple manner? Are the necessary scoring instructions
available? A negative response to any of these questions
could direct you to go to another, more feasible, instrument.
Feasibility often makes or breaks a decision about a candi-
date measure.69
STEP 3: IS IT MEASURING WHAT IT SAYS
IT IS MEASURING? (TRUTH)
Does the instrument measure what it says it will measure?
We divide this into three areas: content, face, and construct
validity. Content validity appraises the items and domains
of a scale. Have the authors covered what McHorney and
Tarlov66 call the breadth and the depth of the concept, that
is, all the important areas, but also enough depth to capture
the range of experience of the patients? Face validity is an
appraisal of the general direction of the scale—will it hit
the target? Are the response options organized in a logical
direction for high and low levels of this attribute? Does the
scoring make sense?
The stage of construct validity is the dividing point
in the decision process between data-free and data-based
appraisal. Up until now, the appraisal is done by look-
ing at the instrument and its manuals. In construct valid-
ity, we begin to explore data to see if the numeric scores
arising from the instrument make sense. Basic construct
validity should be established regardless of the purpose.
Sometimes it is de-emphasized in evaluative instruments;
however, responsiveness without knowing if the instru-
ment is measuring the target seems misplaced. We place
it before the purpose-specific properties to emphasize its
need as a basis. Construct validity is generally measured
by comparisons with other similar scales or related con-
structs (i.e., high and low levels of pain and function) to
see if the numeric scores are behaving in the way they
should if this were a valid measure of the target con-
cept. Theoretic situations are set up before analysis, the
direction and magnitude of the expected relationship are
declared, and then the relationship is tested.68,73 Com-
parisons also should be made between groups known to
differ (high versus low severity) or with scales where no
469
relationship is found—again based on an a priori the-
ory—to see if the candidate measure behaves according
to the theory. These add to the evidence that the instru-
ment is measuring what it is supposed to measure.68 If the
evidence is unavailable or is not in the intended popula-
tion, you have a choice of abandoning the measure or
doing a study to create that evidence and then continu-
ing to advance.
STEP 4: PURPOSE-SPECIFIC EVIDENCE IN YOUR
POPULATION
After getting a general sense of the construct validity of the
instrument, the next step is to address the specific attributes
needed for the instrument to function to meet your measure-
ment need (discrimination component of the OMERACT
filter).69 More attention is now paid to the intended purpose
and the intended population and the properties of consis-
tency in measurement (reliability—obtaining the same score
in different settings) and additional validity (cross-sectional
again or responsiveness or sensitivity to change).
Descriptive Purpose
Descriptive outcomes often are used to classify individuals
as to severity of condition or to identify them by a prog-
nostic group. In health outcome assessment, descriptive
instruments are needed to classify individuals as respond-
ers or as being in a low disease activity state or remission.
An instrument needs to be precise, that is, the observed
score is very close to the true score with low error. This is
estimated by the internal consistency of a multi-item scale
or questionnaire and Cronbach alpha coefficients or Kuder
Richardson 20 if the scale is dichotomous (yes/no). Internal
consistency is a feature of a scale with many items measur-
ing the same thing—are the responses similar across items
within the instrument? It is not a feature of a scale con-
taining weighted sums of different attributes, such as disease
activity measures.37
If more than one person will be gathering the data, inter-
rater/observer reliability should be measured and quantified
with an intraclass correlation coefficient (ICC) for continu-
ous measures or a weighted kappa for ordered categories.74
There are different types of ICC depending on the model
used for the variance estimates; the type of ICC should be
named.74 The ICC and weighted kappa measure the com-
parability of actual numeric scores and are preferred over
correlation coefficients that look only for trends and not
a direct match in number values. Cutoffs are always chal-
lenging, but in general, reliability (including test-retest)
should be at minimum 0.7566,73 for group level analyses,
and for describing an individual patient, it should be 0.90
to 0.95.66,73 The internal consistency reliability can be con-
verted back into the scale score by calculating the precision
limits—using 95% limits, the true score lists somewhere
within 1.96 × s[1 − r]1/2 where r = internal consistency and
s = standard deviation. This calculation tells us the range
within which the true score for an individual can be found.
If it is too wide (reliability too low), it is impractical to use
that instrument.
Construct validity is revisited for the descriptive instru-
ment, but with more attention to looking for evidence close
 470 BEAToN  |  Assessment of Health Outcomes
to the intended application. If the goal is to measure high
versus low health, the sample should be divided into known
groups with high and low health according to another
accepted opinion, and then this scale is tested against it.
The image of a window can help here in selecting compara-
tors to use for testing. What else gives a bit (or more) of
overlap with the target view? How much correspondence is
expected between scores? For good construct validity, this
a priori hypothesized relationship should be recreated with
data, whether that be a strong correlation or no correlation
at all. An instrument or measure is never universally valid
and requires ongoing testing to improve understanding of
the scores in different situations.
Evaluative Purpose
In evaluative measures, the intent of the study is to focus on
the amount of change over time. Many clinical trials are doing
this and comparing results between treatment and control
groups. Interobserver reliability is important if more than one
measurer is to be involved. The hallmark of a good evaluative
measure relates, however, to time: First, do the scores remain
the same when the target concept has not changed over time
(test-retest reliability)? Second, when the concept changes,
does the score on the instrument/measure change as well?
Test-retest reliability requires two administrations of
the measure over a time when no change has occurred.
This may be easier said than done sometimes, but the
authors should justify their design and how they ensured
no change had occurred. Similar to interobserver reliabil-
ity, the ICC is the preferred statistic for continuous scores,
and weighted kappa, its equivalent, is preferred for cate-
gorical scores. The cutoffs are the same, and a coefficient
can be converted into a “minimal detectable change”75
as 1.96 × s(2[1 − r])1/2, where s = standard deviation and
r = test-retest reliability (ICC).66,75 Ninety-five percent of
subjects who are stable have change scores less than this
value; a change greater than this is not likely to occur in a
stable patient, only in a changing one. It becomes a lower
boundary of meaningful change—anything below that
could be day-to-day fluctuations in scores.
Responsiveness—the accurate detection of change when
it has occurred—is sometimes best thought of as longitudinal
construct validity. Similar to construct validity, responsive-
ness depends on an a priori theoretic relationship—one in
which the attribute is changing over time. Often the focus
is on the amount of change picked up, rather than the type
or amount of change that had occurred. A large change is
not useful if we were expecting a small one; rather it suggests
noise. The construct embedded in a study of responsiveness
should be described carefully and should be a clear match with
the intended application (measurement need). If the goal is
to detect change in a clinical trial, it is important to assess the
instrument’s ability to detect the difference in change between
treatment and control groups. If the goal is to detect change
in a cohort, it might be more useful to examine change in a
single group perhaps in a treatment of known efficacy (hip
replacement) or in subjects who rated themselves as improved
on an external anchor (global index of change).
Responsiveness is summarized with statistics of sig-
nal (change) over noise (error), such as the standardized
response mean (mean change/standard deviation of change),
t ­statistic (mean change/standard error), and effect size (mean
change over standard deviation of baseline)74; each can be
adapted to quantify the relative change between treatment
and control groups.53,76 Deyo and Centor77 also described
the correlational approach (correlate change and another
indicator of change) and the receiver operator curve approach
(various change scores against external “gold standard” that
the person has changed) where the area under the curve is
a summary statistic.77 The numeric summaries of responsive-
ness, such as effect sizes or areas under the curve, should cor-
respond to the type of change expected (a priori theory). A
large effect size or area under the curve does not mean an
instrument is “responsive.” It should correspond with the
change anticipated in the study—small or large. Comparisons
of the effect sizes are helpful if different instruments are being
compared in the same study, as done by Buchbinder and col-
leagues53 or by Verhoeven and coworkers,76 who focused on
responsiveness in early rheumatoid arthritis. Responsiveness
is a highly contextualized property, and the same instrument
may not be responsive in another situation (e.g., early versus
late disease, osteoarthritis versus rheumatoid arthritis).73
STEP 5: INTERPRETABILITY OF SCORES
The final step, often deemed the most elusive,78 is the inter-
pretability of the scores.
Benchmarking States
What is the meaning of a score of 2/10 on a pain score? Is it
a good outcome? The meaning of different scores on an out-
come assessment is used for classifying subjects at the begin-
ning of a trial and at the end point. To do this, comparisons
are made to other known health states—­severity indices,
ability to work, self-rating as mild.79 Gradually, enough
trends might be seen across different scenarios to gain confi-
dence in the meaning of “good” or “mild.”80,81 In rheumatol-
ogy, we see the emergence of low disease activity states82,83
or patient acceptable symptom states84 or remission criteria
with the DAS2838 as thresholds below which subjects are
considered to be in an acceptable state (either tolerable
symptoms or disease activity where it does not require medi-
cation changes). At this point, these thresholds are being
established, and similar to change thresholds, we may find
variability in the values38 that need to be sorted out with
methodologic work and application in clinical practice.
Changes in State
The second type of interpretability concerns change scores.
American College of Rheumatology Response Criteria.
The American College of Rheumatology took the core
set measures and determined that if one observed an X%
change in joint count and in swollen joint count and in at
least three other areas—erythrocyte sedimentation rate or
C-reactive protein, physician global, patient global, pain,
or physical disability—one had a clinical response, and the
individual would be classified as a responder. The percent
is usually 20%, but 50% and 70% have been considered.
The ACR20 is widely used, catches responses across a
wide variety of domains, and discriminates well in clinical
 PART 4  |  BROAD ISSUES IN THE APPROACH TO RHEUMATIC DISEASES
trials76; however, it is currently being revalidated owing to
the changing nature of rheumatoid arthritis and its care.85
Minimal Clinically Important Differences and Improve-
ments. Defining the threshold of change above which an
individual has had an “important” shift in outcome is what
Kirwan78 has described as the “elusive crock of gold at the
end of the rainbow.” Nevertheless, important advances have
been made. There are many sources of variation in score,
including the method used, the baseline severity, and the
type of change to be sought.86,87 In 2000, Wells and cowork-
ers88 described nine different methods for deriving minimal
clinically important differences from the literature. Some
use distributional cutoffs (½ standard deviation, or effect
size of 0.2 or 0.5),89 which have been criticized as lacking
any meaningful anchor. Other methods depend on some
external anchor that important improvement has occurred,
but are sometimes challenged by the dependence on that
anchor and the perspective of the individual who deter-
mines it (patient, physician, third-party payer). Minimal
clinically important differences repeatedly have been shown
to vary with baseline state90,91 and with improvement versus
deterioration.92 Tubach and associates93 changed the term
to minimal clinically important improvement and looked only
at improvement. Minimal clinically important differences
vary depending on the context of measurement. You need
to plan on working with a range of values,42,86,87 to make sure
the measurement situation is similar to your own (severity,
timing, type of intervention), and to build confidence with
congruence in minimal clinically important differences from
across methods if you can achieve that.
Combined Approaches: Change and State
An attractive, although often overlooked, option is com-
bining change and state. In 1996, EULAR defined clini-
cal response as a change in DAS28 score of more than 1.2
(change) plus a final DAS28 score of less than 2.4 (final
state).36 Jacobson and colleagues94 did the same in defining
response to psychotherapy; change greater than error was
used (minimal detectable change mentioned earlier) plus a
final “normal” state. Studies from the patient’s perspective
have often reflected the same thing.80,81,95 Treatment needs to
induce a change, but perhaps it also needs to land patients
in a healthy state to make them feel better.
The approaches described previously focus on interpreta-
tion at the level of the individual, perhaps for use in clinical
practice or in a response-type analysis of a clinical trial or eco-
nomic appraisal (% responder). Verhoeven and ­coworkers76
showed that the same instrument may not perform equally
well in a responder type of analysis and a group level change.
At each stage of this appraisal, there is an element of
judgment. It is likely there will never be perfect evidence
across all stages. The user needs to assess the potential risk
of accepting less than ideal evidence or abandon the scale.
Users also may create the evidence, however, by doing it
themselves. An instrument that makes it through this
appraisal is likely a good fit with the measurement need.
Anything short of that could lead to error and be prone to
misinterpretation. By working from left to right, scales that
are not targeting the right concept or are impractical to use
in the intended setting can be eliminated quickly before
471
reviewing the literature extensively for the measurement
properties.
SECTION 6: AREAS OF GROWTH IN HEALTH
OUTCOME ASSESSMENT
Item Response Theory
Users of outcomes in arthritis come across item response
theory (IRT) and computer adaptive testing (CAT). These
terms relate to newer methods of ordering and calibrating
items on a scale so that there is equal meaning in score incre-
ments across the scale. Most of our outcomes were developed
in “classic test theory” (internal consistency, summed scores
without weights). There are two schools within IRT: Rasch,
which fixes the parameters and assesses if items in a scale fit
or do not fit that model, and IRT itself, which fits a model
to the data, rather than the reverse. IRT and Rasch are often
presented as conflicting schools, but they are both working
toward an item calibration that allows more accuracy and
precision. In the future, direct comparisons may reveal their
similarities and differences in practical ways. The weights
are cumbersome to apply for the clinician, but can be eas-
ily integrated into a computer-based scoring system for easy
data entry and CAT. CAT chooses items based on the previ-
ous set of responses and uses the fewest number of items to
reach a precise score skipping easier items if confident the
subject can do the harder ones. This streamlined scoring is
quite attractive, but there are some limitations. It depends
on technology that may not at this time be available in
every setting. It also may be influenced by differential item
functioning, which means an item might change weight, or
order, in certain subgroups and necessitates more complex
weights. An example of differential item functioning would
be putting on a pullover sweater. It is a hard task if you have
shoulder pain, but pretty easy if you have a hand problem
and would require a different weight.
The National Institutes of Health is currently funding
PROMIS (Patient Reported Outcomes Measurement Infor-
mation System; available at http://www.nihpromis.org/) to
develop a CAT system (currently based on a two-parameter
graded response model IRT) for common chronic diseases,
including arthritis.96 Several measures have been pooled
into a large database and are being refined and rescaled at
the time of this writing. Well-known measures, such as the
HAQ, also will be used to allow for cross-calibration with
the newer items. All findings will be reported on the PRO-
MIS website, as will access to the scoring algorithms.
Use of Technology in Health Outcomes Assessment
In addition to enabling efforts such as PROMIS to develop
CAT systems, information technology has changed many
aspects of health outcome assessments. Streamlined, cus-
tomized assessments can be set up on the Internet or on a
stand-alone computer with interfaces such as touch screen,
light pens, or “point and click.” Patients can complete the
questionnaires at home, at the clinic, on their PDA, or
on a tablet. Language and literacy issues can be overcome
with talking screens. Scoring becomes instantaneous,
and reports can be printed immediately summarizing the
scored results in time for the clinical visit.97,98 ­Comparisons
 472 BEAToN  |  Assessment of Health Outcomes
between touch screens and traditional paper and pen-  
cils are promising, and the acceptability by patients with
arthritis is good.97-99 New technology means that health  
outcome assessment can become part of the patient-
­clinician ­experience and facilitate the ability of the clini-
cian to monitor the patient’s health.97  
 
Adaptation to an Ongoing Disease
This chapter has focused on the measurement of health states  
and their interpretation over time. Individuals with chronic
diseases adapt to ongoing disease with behavioral strategies or
cognitive reframing of their situation.100 In some circles, this  
is adjustment95; in others, it is response shift.101 The challenge
in health outcomes assessment is to tell when a state is chang-
ing only because of adaptation and not the intervention. In
many situations, we try to induce adaptation, or cognitive
reframing, and it can be constructive. It does create a bias
in measurement,101 however, and a challenge to the health
outcome assessor. Numerous groups are researching how to
incorporate adaptation into health outcome assessments.
SUMMARY
There is considerable room for improvement in health out-
come assessment in rheumatology, despite the work done to
date. A battery of instruments have been developed, many of
which exhibit the measurement properties described in this
chapter and meet the challenge of a changing arthritis tar-
get (less severe, earlier disease), and several more measures
are being considered for membership in core sets to capture a
comprehensive view of the burden of arthritis. We are on the
brink of deciding on the role to be played by IRT and CAT
in widespread care settings. Despite progress in assigning a
numeric value to a complex health state, however, we are now
struggling with the back-translation—what does the numeric
score mean in the real patient world. It is not always a simple
translation from questionnaire score to clinical meaning.
Health outcome assessment is well advanced in arthritis care,
and we should recognize the years of work and commitment of
many professional and patient/consumer groups. Advances will
continue in the use of technology, the breadth and depth of
outcomes, and the quality of measurement to keep pace with
the needs of patients, clinicians, and researchers.
Acknowledgments
Dorcas Beaton is supported by a New Investigators Award through the
Canadian Institutes of Health Research. Peter Tugwell holds a Canada
Research Chair.
The authors would like to thank Ms. Taucha Inrig, Dr. Claire
­Bombardier, Dr. Fred and Mrs. Janet Krieger, Mr. William Francis, and
the OMERACT executive for their help with this manuscript, and
Dr. M. Ward, whose chapter in the seventh edition of Kelley’s Textbook of
Rheumatology was a helpful guide.
REFERENCES
  1. Relman AS: Assessment and accountability: The third revolution in
medical care. N Engl J Med 319:1220-1222, 1988.
  2. Last JM: A Dictionary of Epidemiology. New York, Oxford University
Press, 1988.
  3. Prevoo MLL, Van’t Hof MA, et al: Modified disease activity scores
that include twenty-eight-joint counts: Development and ­validation
in a prospective longitudinal study of patients with rheumatoid
arthritis. Arthritis Rheum 38:44-48, 1995.
4. Fries JF, Spitz PW, Young DY: The dimensions of health outcomes:
The Health Assessment Questionnaire, Disability and Pain Scales.
J Rheumatol 9:19203, 1982.
5. Ware JE Jr, Sherbourne CD: The MOS 36-Item Short-Form Health
Survey (SF-36), I: Conceptual framework and item selection. Med
Care 30:473-483, 1992.
6. Fries JF: The hierarchy of outcome assessment. J Rheumatol 20:
546-547, 1993.
7. Wolfe F, Lassere M, van der Heijde D, et al: Preliminary core set
of domains and reporting requirements for longitudinal observa-
tional studies in rheumatology. J Rheumatol 26:484-489, 1999.
8. van der Heijde D, van der Linden S, Bellamy N, et al: Which
domains should be included in a core set for endpoints in ankylos-
ing spondylitis? Introduction to the ankylosing spondylitis module of
OMERACT IV. J Rheumatol 26:945-947, 1999.
9. Gladman DD, Mease PJ, Healy P, et al: Outcome measures in psori-
atic arthritis (PsA). J Rheumatol 34:1159-1166, 2007.
10. Gladman DD, Mease PJ, Strand V, et al: Consensus on a core set of
domains for psoriatic arthritis. OMERACT 8 PsA Module Report.
J Rheumatol 34:1167-1170, 2007.
11. Sambrook PN, Cummings SR, Eisman JA, et al: Guidelines of osteo-
porosis trials (workshop report). J Rheumatol 24:1234-1236, 1997.
12. Gladman DD, Strand V, Mease PJ, et-al: OMERACT 7 psoriatic
arthritis workshop: Synopsis. Ann Rheum Dis 64:ii-115-ii-116, 2005.
13. Bellamy N, Kirwan J, Boers M, et al: Recommendations for a core set
of outcome measures for future phase III clinical trials in knee, hip,
and hand osteoarthritis: Consensus development at OMERACT III.
J Rheumatol 24:799-802, 1997.
14. Smolen JS, Strand V, Cardiel M, et al: Randomized clinical trials and
longitudinal observational studies in systemic lupus erythematosus:
Consensus on a preliminary core set of outcome domains. J Rheuma-
tol 26:504-507, 1999.
15. Boers M, Tugwell P, Felson DT, et al: World Health Organization
and International League of Associations for Rheumatology core
endpoints for symptom modifying antirheumatic drugs in rheumatoid
arthritis clinical trials. J Rheumatol 21:86-89, 1994.
16. Felson DT, Anderson JJ, Boers M, et al: The American College of
Rheumatology preliminary core set of disease activity measures for
rheumatoid arthritis clinical trials. The Committee on Outcome
Measures in Rheumatoid Arthritis Clinical Trials. Arthritis Rheum
36:729-740, 1993.
17. van der Heijde D, Landewe R: Selection of a method for scoring radio-
graphs for ankylosing spondyolitis clinical trials, by the Assessment
in Ankylosing Spondylitis working groups (ASAS) and OMERACT.
J Rheumatol 32:2048-2049, 2005.
18. Guidelines of osteoporosis trials (workshop report). J Rheumatol
24:1234–1236, 1997.
19. Strand V, Gladman DD, Isenberg D, et al: Outcome measures to be
used in clinical trials in systemic lupus erythematosus. J Rheumatol
26:490-497, 1999.
20. Patrick DL, Deyo RA: Generic and disease-specific measures in
assessing health status and quality of life. Med Care 27(Suppl):
S217-S232, 1989.
21. Bergner M, Bobbitt RA, Pollard WE, et al: The sickness impact pro-
file: Validation of a health status measure. Med Care 14:57-67, 1976.
22. Ware JE Jr: SF-36 health survey update. Spine 25:3130-3139, 2000.
23. Ware JE Jr, Snow KK, Kosinski M, et al: SF-36 Health Survey Man-
ual and Interpretation Guide. Boston, The Health Institute, 1993.
24. Beaton DE, Bombardier C, Hogg-Johnson SA: Measuring health in
injured workers: A cross-sectional comparison of five generic health
status instruments in workers with musculoskeletal injuries. Am J Ind
Med 29:618-631, 1996.
25. Beaton DE, Hogg-Johnson S, Bombardier C: Evaluating changes in
health status: Reliability and responsiveness of five generic health
status measures in workers with musculoskeletal disorders. J Clin Epi-
demiol 50:79-93, 1997.
26. Visser MC, Fletcher AE, Parr G, et al: A comparison of three qual-
ity of life instruments in subjects with angina pectoris: The Sickness
Impact Profile, the Nottingham Health Profile, and the Quality of
Well Being Scale. J Clin Epidemiol 47:157-163, 1994.
27. Revicki DA, Kaplan RM: Relationship between psychometric and
utility-based approaches to the measurement of health-related qual-
ity of life. Qual Life Res 2:477-487, 1993.
28. Feeny D: Preference-based measures: utility and quality-adjusted life
years. In Fayers P, Hays R (eds): Assessing Quality of Life in Clinical
Trials, 2nd ed. New York, Oxford University Press, 2005, pp 405-429.
 PART 4  |  BROAD ISSUES IN THE APPROACH TO RHEUMATIC DISEASES
29. Farrar JT, Portenoy RK, Berlin JA, et al: Defining the clinically
important difference in pain outcome measures. Pain 88:287-294,
2000.
30. Garrett S, Jenkinson T, Kennedy LG, et al: A new approach to defin-
ing disease status in ankylosing spondylitis: The BATH Ankylosing
Spondylitis Disease Activity Index. J Rheumatol 21:2286-2291, 1994.
31. Fries JF: The hierarchy of quality-of-life assessment, the Health Assess-
ment Questionnaire (HAQ), and issues mandating development of a
toxicity index. Controlled Clinical Trials 12:106S-117S, 1991.
32. Meenan RF, Gertman PM, Mason JH: Measuring health status in
arthritis: The Arthritis Impact Measurement Scales. Arthritis Rheum
23:146-152, 1980.
33. Meenan RF, Mason JH, Anderson JJ, et al: Aims2: The content and
properties of a revised and expanded arthritis impact measurement
scales health status questionnaire. Arthritis Rheum 35:1-10, 1992.
34. Bellamy N, Buchanan WW, Goldsmith CH, et al: Validation study
of WOMAC: A health status instrument for measuring clinically-
important patient-relevant outcomes following total hip or knee
arthroplasty in osteoarthritis. J Orthop Rheum 1:95-108, 1988.
35. Bellamy N, Campbell J, Haraoui B, et al: Clinimetric properties of the
AUSCAN osteoarthritis hand index: An evaluation of reliability, valid-
ity and responsiveness. Osteoarthritis Cartilage 10:863-869, 2002.
36. Van Gestel AM, Prevoo MLL, Van’t Hof MA, et al: Development
and validation of the European League Against Rheumatism response
criteria for rheumatoid arthritis. Arthritis Rheum 39:34-40, 1996.
37. Vrijhoef HJM, Diederiks JPM, Spreeuwenberg C, et al: Applying low
disease activity criteria using the DAS28 to assess stability in patients
with rheumatoid arthritis. Ann Rheum Disease 62:419-422, 2003.
38. Aletaha D, Ward MM, Machold KP, et al: Remission and active
disease in rheumatoid arthritis: Defining criteria for disease activity
states. Arthritis Rheum 52:2625-2636, 2005.
39. Lassere M, van der Heijde D, Johnson K, et al: Robustness and gen-
eralizability of smallest detectable difference in radiological progres-
sion. J Rheumatol 28:911-913, 2001.
40. Ravaud P, Giraudeau B, Auleley GR, et al: Assessing smallest detect-
able change over time in continuous structural outcome measures:
Application to radiological change in knee osteoarthritis. J Clin Epi-
demiol 52:1225-1230, 1999.
41. Lassere M, Johnson K, Van Santen S, et al: Generic patient self-
report and investigator report instruments of therapeutic safety and
tolerability. J Rheumatol 32:2033-2036, 2005.
42. U.S. Department of Health and Human Services Food and Drug
Administration Center for Drug Evaluation and Research (CDER):
Guidance for industry: Patient-reported outcome measures: Use
in medical product development to support labeling claims: Draft
guidance. Available at: http://www.fda.gov/cder/gdlns/prolbl.htm.
Accessed November 3, 2006.
43. Woodworth T, Furst DE, Alten R, et al: Standardizing assessment and
reporting of adverse effects in rheumatology clinical trials, II: Rheu-
matology Common Toxicity Criteria v2.0. J Rheumatol 34:1401-
1414, 2007.
44. Kristjansson E, Tugwell PS, Wilson AJ, et al: Development of the
effective musculoskeletal consumer scale. J Rheumatol 34:1392-
1400, 2007.
45. Escorpizo R, Bombardier C, Boonen A, et al: Worker productivity
outcome measures in arthritis. J Rheumatol 34:1372-1380, 2007.
46. Lerner D, Amick BC III, Rogers WH, et al: The work limitations
questionnaire. Med Care 39:72-85, 2001.
47. Gignac MAM, Badley EM, Lacaille D, et al: Managing arthritis and
employment: Making arthritis-related work changes as a means of
adaptation. Arthritis Care Res 51:909-916, 2004.
48. Gilworth G, Chamberlain AM, Harvey A, et al: Development of
a work instability scale for rheumatoid arthritis. Arthritis Care Res
49:349-354, 2003.
49. Backman C, Kennedy SM, Chalmers A, et al: Participation in paid
and unpaid work by adults with rheumatoid arthritis. J Rheumatol
31:47-57, 2004.
50. Tugwell P, Bombardier C, Buchanan WW, et al: The MACTAR
Patient Preference Disability Questionnaire—an individualized
functional priority approach for assessing improvement in physi-
cal disability in clinical trials in rheumatoid arthritis. J Rheumatol
14:446-451, 1987.
51. Buchbinder R, Bombardier C, Yeung M, et al: Which outcome mea-
sures should be used in rheumatoid arthritis clinical trials? Clinical
and quality-of-life measures’ responsiveness to treatment in a ran-
domized controlled trial. Arthritis Rheum 38:1568-1580, 1995.
473
52. O’Boyle CA, Hofer S, Ring L: Individualized quality of life. In ­Fayers
P, Hays R (eds): Assessing Quality of Life in Clinical Trials: Methods
and Practice, 2nd ed. New York, Oxford University Press, 2005,
pp 225-242.
53. Buchbinder R, Bombardier C, Yeung M, et al: Which outcome
measures should be used in rheumatoid arthritis clinical trials?
Arthritis Rheum 38:1568-1580, 1995.
54. Solomon DH, Bates DW, Horsky J, et al: Development and valida-
tion of a patient satisfaction scale for musculoskeletal care. Arthritis
Care Res 12:96-100, 1999.
55. Hudak PL, McKeever PD, Wright JG: Understanding the meaning of
satisfaction with treatment outcome. Med Care 42:718-725, 2004.
56. Jette AM, Haley SM: Contemporary measurement technique for
rehabilitation outcome assessment. J Rehabil 37:339-345, 2005.
57. Jette AM, Keysor JJ: Disability models: Implications for arthritis
exercise and physical activity interventions. Arthritis Care Res 49:
114-120, 2003.
58. Wilson IB, Cleary PD: Linking clinical variables with health-related
quality of life: A conceptual model of patient outcomes. JAMA
273:59-65, 1995.
59. Nagi SZ: A study in the evaluation of disability and rehabilitation
potential. Am J Public Health 54:1568-1579, 1964.
60. Verbrugge LM, Jette AM: The disablement process. Soc Sci Med
38:1-14, 1994.
61. World Health Organization: International Classification of Function-
ing, Disability and Health. Geneva, World Health Organization, 2001.
62. Arts DGT, Keizer NF, Scheffer G: Defining and improving data qual-
ity in medical registries: A literature review, case study and generic
framework. J Am Med Inform Assoc 9:600-611, 2002.
63. Stucki G, Boonen A, Tugwell P, et al: The World Health Organi-
sation International Classification of Functioning, Disability and
Health (ICF): A conceptual model and interface for the OMERACT
process. J Rheumatol 34(3):600-606, 2007.
64. Jette AM: Toward a common language for function, disability and
health. Phys Ther 86:726-734, 2006.
65. Jette AM, Keysor JJ: Uses of evidence in disability outcomes and
effectiveness research. Milbank Q 80:325-345, 2002.
66. McHorneyCA, Tarlov AR: Individual patient monitoring in clini-
cal practice: Are available health status surveys adequate? Qual
Life Res 4:293, 1995.
67. Lohr KN, Aaronson NK, Alonso J, et al: Evaluating quality-of-life
and health status instruments: Development of scientific review cri-
teria. Clin Therap 18:979-992, 1996.
68. McDowell I, Jenkinson C: Development standards for health mea-
sures. J Health Serv Res Policy 1:238-246, 1996.
69. Boers M, Brooks P, Strand V, et al: The OMERACT Filter for out-
come measures in rheumatology. J Rheumatol 25:198-199, 1998.
70. Kane RA, Kane RL: Assessing the Elderly: A Practical Guide to mea-
surement. Toronto, Lexington Books, 1981, pp13-17.
71. Bergner M: Health status measures: An overview and guide for selec-
tion. Ann Rev Public Health 8:191-210, 1987.
72. Law M: Measurement in occupational therapy: Scientific criteria for
evaluation. Can J Occup Ther 54:133-138, 1987.
73. Scientific Advisory Committee of the Medical Outcomes Trust:
Assessing health status and quality of life instruments: Attributes
and review criteria. Qual Life Res 11:193-205, 2002.
74. Hays RD, Revicki D: Reliability and validity (including responsive-
ness). In Fayers P, Hays R (eds): Assessing Quality of Life in Clinical
Trials: Methods and Practice, 2nd ed. New York, Oxford University
Press, 2005, pp 25-39.
75. Stratford PW, Binkley JM: Applying the results of self-report mea-
sures to individual patients: An example using the Roland-­Morris
Questionnaire. J Orthop Sports Physical Ther 29:232-239,
1999.
76. Verhoeven A, Boers M, van der Linden S: Responsiveness of the core
set, response criteria, and utilities in early rheumatoid arthritis. Ann
Rheum Dis 59:966-974, 2000.
77. DeyoRA, Centor RM: Assessing the responsiveness of functional
scales to clinical change: An analogy to diagnostic test perfor-
mance. J Chronic Dis 39:897-906, 1986.
78. Kirwan J: Minimum clinically important difference: The crock of
gold at the end of the rainbow? J Rheumatol 28:439-444, 2001.
79. Deyo RA, Carter WB: Strategies for improving and expand-
ing the application of health status measures in clinical set-
tings: A researcher-developer viewpoint. Med Care 30(5 Suppl):
MS176-MS186, 1992.
 474 BEAToN  |  Assessment of Health Outcomes
80. Tubach F, Dougados M, Falissard B, et al: Feeling good rather than
feeling better matters more to patients. Arthritis Care Res 55:
526-530, 2006.
81. Beaton DE, Tarasuk V, Katz JN, et al: Are you better? A qualitative
study of the meaning of being better. Arthritis Care Res 7:313-320,
2001.
82. Boers M, Anderson JJ, Felson D: Deriving an operational definition
of low disease activity state in rheumatoid arthritis. J Rheumatol
30:1112-1114, 2003.
83. Tubach F, Wells GA, Ravaud P, et al: Minimal clinically important
difference, low disease activity state and patient acceptable symptom
state: Methodological issues. J Rheumatol 32:2025-2029, 2005.
84. Tubach F, Ravaud P, Baron G, et al: Evaluation of clinically relevant
states in patient reported outcomes in knee and hip osteoarthrits:
The patient acceptable symptom state. Ann Rheum Dis 64:34-37,
2005.
85. Felson DT, Furst DE, Boers M: Rationale and strategies for reeval-
uating the ACR20. J Rheumatol 34:1184-1187, 2007.
86. Beaton DE, Boers M, Wells GA: Many faces of the minimal clini-
cally important difference (MCID): A literature review and direc-
tions for future research. Curr Opin Rheumatol 14:109-114,
2002.
87. Hays RD, Woolley JM: The concept of clinically meaningful differ-
ence in health-related quality of life research. PharmacoEconomics
18:419-423, 2000.
88. Wells GA, Beaton DE, Shea B, et al: Minimal clinically important
differences: Review of methods. J Rheumatol 28:406-412, 2001.
89. Norman GR, Sloan JA, Wyrwich KW: Interpretation of changes in
health-related quality of life: The remarkable universality of half a
standard deviation. Med Care 41:582-592, 2003.
90. Salaffi F, Stancati A, Silvestri CA, et al: Minimal clinically impor-
tant changes in chronic musculoskeletal pain intensity measures on a
numerical rating scale. Eur J Pain 8:283-291, 2004.
91. Stucki G, Daltroy L, Katz JN, et al: Interpretation of change scores in
ordinal clinical scales and health status measures: The whole may not
be equal to the sum of the parts. J Clin Epidemiol 49:711-717, 1996.
92. Angst F, Aeschlimann A, Stucki G: Smallest detectable and minimal
clinically important differences of rehabilitation intervention with
their implications for required sample sizes using WOMAC and SF-
36 quality of life measurement instruments in patients with osteo-
arthritis of the lower extremities. Arthritis Care Res 45:384-391,
2001.
93. Tubach F, Ravaud P, Baron G, et al: Evaluation of clinically relevant
changes in patient reported outcomes in knee and hip osteoarthritis:
The minimal clinically important improvement. Ann Rheum Dis
64:29-33, 2005.
94. Jacobson NS, Roberts LJ, Berns SB, et al: Methods for defining and
determining the clinical significance of treatment effects: Descrip-
tion, application, alternatives. J Consult Clin Psychol 67:300-307,
1999.
95. Norman G: Hi! How are you? Response shift, implicit theories and
differing epistemologies. Qual Life Res 12:249, 2003.
96. National Institutes of Health: Patient Reported Outcome Measure-
ment Information System (PROMIS) network. 2006. Available at:
http://www.nihpromis.org/.
97. Athale N, Sturley A, Koczen Z, et al: A web-compatible instrument
for measuring self-reported disease activity in arthritis. J Rheumatol
31:223-228, 2004.
98. Fransen J, Stucki G, Twisk J, et al: Effectiveness of a measurement
feedback system on outcome in rheumatoid arthritis: A controlled
clinical trial. Ann Rheum Dis 62:624-629, 2003.
99. Bischoff-Ferrari HF, Vandechend M, Bellamy N, et al: Validation
and patient acceptance of a computer touch screen version of the
WOMAC 3.1 osteoarthritis index. Ann Rheum Dis 64:80-84, 2004.
100. Shaul MP: From early twinges to mastery: The process of adjustment in
living with rheumatoid arthritis. Arthritis Care Res 8:290-297, 1995.
101. Schwartz C, Sprangers M, Fayers P: Response shift: You know it’s
there but how do you capture it? Challenges for the next phase of
research. In Fayers P, Hays R (eds): Assessing Qualify of Life in Clini-
cal Trials: Methods and Practice, 2nd ed. New York, Oxford Univer-
sity Press, 2005, pp 275-290.