Accuracy of Observations and Measurements 2 DEC 2015

Jose Luisito A. Zulueta, M.D.

OUTLINE C. INTRA-INDIVIDUAL VARIATION (WITHIN SUBJECT)
I. Sources of Variation Chances of error referring to changes in the condition of the
a. Intra-observer variation subject being observed
b. Inter-observer variation Same person, different readings
Ex: taking blood pressure before and after patient has
c. Intra-individual variation
taken his/her medication
d. Inter-individual variation
e. Instrument variation D. INTER-INDIVIDUAL VARIATION (BETWEEN
II. Accuracy and Precision SUBJECTS)
III. 2x2 Contingency Table Chances of error referring to the differences in
IV. Sensitivity and Specificity characteristics among different subjects being observed
V. Predictive Value Different subjects react in different ways
a. Positive Predictive Value Ex. some men would react differently when a beautiful
b. Negative Predictive Value nurse takes the BP
VI. Prevalence
VII. Clinical Applications E. INSTRUMENT VARIATION
VIII. Sample 2x2 Contingency Table Computations Chances of error referring to technical differences in
IX. Exercise 3 Part I instrument devices used to gather info/measurements
X. Exercise 3 Part II If the instrument is used for a long time, variations may
XI. Sample Quiz occur
Variations between types of instruments
OBJECTIVES
Ex. difference between aneroid and digital
At the end of the lecture, the student should be able to: sphygmomanometer
1. Identify the various types and sources of variations of
statistical data How to eliminate errors and variations?
2. Identify errors in measurement
Standardized method of collecting measurements
3. Formulate ways of increasing the accuracy of Calibrate the instruments
observation
4. Determine the accuracy of observations and ISO Instrument, subject, observer
measurements
REFERENCES II. ACCURACY AND PRECISION

A. ACCURACY
Dr. Zuluetas ppt
The ability of measurement to be correct on the average
Legend:
Remember Previous
Lecturer Book Trans Com ACCURACY = True Positive + True Negative
(Exams) Trans
True Positive + True Negative + False Positive + False Negative

B. PRECISION
I. SOURCES OF VARIATION
The ability of a measurement to give the same or a very
A. INTRA-OBSERVER VARIATION (WITHIN OBSERVER) similar result with repeated measurements of the same
thing
Chances of error referring to changes in the condition of the Repeatability or reproducibility
observer analyzing the data
A single observer is performing the measurements
Ex. difference in reading an x-ray before and after
lunch by the same radiologist

B. INTER-OBSERVER VARIATION (BETWEEN

OBSERVERS)
Chances of error referring to differences in characteristics
among different people analyzing the same data
Different observers are taking the measurements
Ex. different x-ray readings between different
radiologists; when using a questionnaire, you have
different strategies in conducting the interview Figure 1. Illustration of Differences in Accuracy and Precision.

Page 1 of 5
Transcribers: Garcia,Gesmundo, Gibe, Gonzales, Gonzalez
Editors: Armada
[EPI][Accuracy of Observations and Measurements]
 III. 2x2 CONTINGENCY TABLE V. PREDICTIVE VALUE

A. POSITIVE PREDICTIVE VALUE

The probability that a subject HAS the disease if the test is
POSITIVE
If the test is positive, what is the probability that the subject
really has the disease?

PPV = True Positive

True Positive + False Positive
Figure 2. 2x2 contingency table.

A - TRUE POSITIVES B. NEGATIVE PREDICTIVE VALUE

o Positive test result, & has disease Probability that the subject DOES NOT have the disease if he
B - FALSE POSITIVES or she tested NEGATIVE
o Positive test result, & no disease If the test is negative, what is the possibility that the subject
o Type 1 error or alpha error (assuming) really has no disease?
C - FALSE NEGATIVES
o Negative test result, & has disease NPV = True Negative_______
o Type 2 error or beta error True Negative + False Negative
D - TRUE NEGATIVES
o Negative test result, & no disease When computing for predictive values, you compute it
horizontally (in the 2x2 table)

VI. RELATIONSHIP BETWEEN DISEASE AND

PREDICTIVE VALUE
Prevalence - the total number of people who have the
disease in the entire population under observation
If there is an increase in the PREVALENCE, true positives
will be affected false positive rate
Importance of SCREENING TESTS: If there is high false
positive rates, use of SCREENING TEST (HIGH sensitivity
and LOW false negative error rate; rule out disease), e.g.
ELISA for HIV, is tolerable because we have
CONFIRMATORY TESTS (HIGH specificity and LOW false
positive error rate; rule in disease).
Figure 3. Sample data for contingency table.
Lower Prevalence = PPV is also low;
Higher Prevalence = PPV is also high
IV. SENSITIVITY AND SPECIFICITY

A. SENSITIVITY PREVALENCE = True Positive + False Negative

The ability of a test to label positive those who truly have the
disease or ability of test to detect disease when the
disease is present
True Positive + False Positive + True Negative + False Negative
Table 1. Sample Data relating PREVALENCE to PPV. Higher specificity
of test = greater PPV.

SENSITIVITY =
+

B. SPECIFICITY
The ability of test to label as negative those who dont have
the disease or
The ability of a test to detect non disease when there is
really no disease

SPECIFICITY = +

When computing for sensitivity and specificity, it is

computed vertically (in the 2x2 table)

Figure 2. Sample Data relating PREVALENCE to NPV. Higher

sensitivity = greater NPV.
Transcribers: Garcia, Gesmundo, Gibe, Gonzales, Gonzalez
Editors: Armada
[EPI][Accuracy of Observations and Measurements] Page 2 of 5
 There is DIRECT relationship between PREVALENCE
and the PREDICTIVE VALUES.
B. CONFIRMATORY TESTS
Used for further analysis of a sample to confirm a positive
or sometimes, negative result
Used to rule in true hypotheses/diagnoses
Must have high degree of specificity and a low false
positive (type I error rate) to ensure that a disease that was
screened positive should really be positive only for the
disease the test is specific and not be positive to other type
of diseases
o SpIn use a highly specific test to rule in theories for
differential diagnoses

Figure 3. Changing prevalence. VIII. SAMPLE 2x2 CONTINGENCY TABLE

COMPUTATIONS

Sensitivity = TP/(TP+FN)
=350/(350+150) x100
= 70%

Specificity =TN/(TN+FP)
= 200/(200+250) x100
= 44%

PPV = TP/(TP+FP)
Figure 4. Effect of Sensitivity. =350/(350+250) x100
= 58%

NPV = TN/(TN+FN)
= 200/(200+150) x100
= 57%

Sensitivity = TP/(TP+FN)
Figure 5. Effect of Change of Specificity. = 100/ (100 + 300) x 100
= 25%
Change in specificity has a greater impact on PPV than
Change in Sensitivity Specificity = TN/(TN+FP)
= 400 / (400 + 200) x 100
VII. CLINICAL APPLICATIONS = 67%

A. SCREENING TESTS PPV=TP/(TP+FP)

Used to detect early disease or risk factors for disease in =100/(100+200) x100
=33%
large numbers of apparently healthy individual
Used to rule out false hypotheses/differential diagnoses NPV = TN/(TN+FN)
Must have high degree of sensitivity and a low false = 400 / (400 + 300) x 100
negative (type II error rate) to ensure that a disease that is = 57%
there is not overlooked
o SnOut use a high sensitivity test to rule out
differential diagnosis

Transcribers: Garcia, Gesmundo, Gibe, Gonzales, Gonzalez

Editors: Armada
[EPI][Accuracy of Observations and Measurements] Page 3 of 5
 IX. EXERCISE 3 - PART I
1. Give your observations on the blood pressure reading
obtained by the readers.
- An increase in activity increases blood pressure
- Blood pressure measurements of readers show
differences but remain within proximity
- Due to physiological differences of subjects (weight,
height, body fat, muscle mass, gender, etc), blood
pressure measurements show variations.
2. What are the possible sources of variations in
observation and measurements? Explain briefly.
- Intra-observer Variation (within observer)
chances of error referring to changes in the condition
of the observer analyzing the data (ex: faulty hearing of
Korotkoff sounds, distractions, failure to use correct
methods)
- Inter-observer (between observers) chances of
error referring to differences in characteristics among
different people analyzing the same data (ex: amount
of experience performed)
- Intra-individual Variation (within subject) chances
of error referring to changes in the condition of the
subject being observed (ex: ability to relax or move,
lack of interest with experiment)
- Inter-individual Variation (between subjects)
chances of error referring to the differences in
characteristics among different subjects being
observed (ex: body size, gender, etc)
- Instrument Variation chances of error referring to
technical differences in instrument/devices used to
gather info/measurements (ex: aneroid vs. digital
sphygmomanometers)
3. How can these sources of variations be minimized?
What are ways of enhancing the accuracy of
measurements?
- Consistent atmosphere while working
- Training of experimenters/testers
- Measurements of data should be accurate
(measurements close to real value) and precise
(consistent results among different measurements)
Screening tests can be used to obtain accurate
and precise diagnosis
- Instrument calibration
X. EXERCISE 3 - PART II
1. How accurate is the Doppler guided biopsy in the diagnosis
of prostate cancer? Show calculations.
Accuracy = (TP + TN) / (TP + TN + FP + FN) x 100
= (19+17) / (19 + 17 + 11 + 3) x 100
= 72%
Sensitivity = TP / (TP + FN) x 100
= 19 / (19 + 3) x 100
= 86.36% = 86%
Specificity = TN / (TN + FP) x 100
= 17 / (17 + 11) x 100
= 60.71% = 61%
The Doppler guided biopsy is 72% accurate in the diagnosis of
prostate cancer.
2. What is the probability of having prostate cancer among
patients with a positive Doppler guided biopsy? Show your
calculation.
Transcribers: Garcia, Gesmundo, Gibe, Gonzales, Gonzalez
Editors: Armada
[EPI][Accuracy of Observations and Measurements]
PPV = TP / (TP + FP) x 100
= 19 / (19 + 11) x 100
= 63.33% = 63%
There is a 63% probability of patients diagnosed as positive by
the Doppler guided biopsy to having prostate cancer.
3. Define the following:
a. Sensitivity
Probability of testing positive when disease is present.
b. Specificity
Probability of testing negative when disease is absent
c. Positive Predictive Value
Probability of having a disease when test is positive
d. Negative Predictive Value
Probability of not having a disease when test is
negative
e. True Positives
Individuals who test positive for disease and actually
have the disease (diseased individuals who test
positive)
f. False Positives
Individuals who test positive for disease but do not
actually have the disease (disease-free individuals who
test positive)
g. True Negatives
Individuals who test negative for disease and do not
have the disease (disease-free individuals who test
negative)
h. False Negatives
Individuals who test negative for disease but actually
have the disease (diseased individuals who test
negative)
i. Type I error / error
Rejecting a null hypothesis when it is in fact, true
j. Type II error / error
Failing to reject a false null hypothesis
4. Explain the influence of the prevalence of the disease on
the number of true positives and true negatives.
- Prevalence of disease in the population tested is the test
itself (sensitivity and specificity). In general, it depends
more on the specificity (and less on the sensitivity) of the
test if disease is present.
o If prevalence is high, positive predictive value is also
high
o If prevalence is low, true positives are low, true
negatives are high, positive predictive value is low.
Prevalence DIRECTLY affects the predictive value of any
test. As prevalence rises, the positive predictive value will
also rise which may lead to a huge discrepancy in the
clinical interpretation of the same test result.
Page 4 of 5
 XI. SAMPLE QUIZ
1. An example of this variation is the different body types and
gender of the subject
a. intra-individual variation
b. inter-individual variation
c. inter-observer variation
d. intra-observer variation

2. The ability of a test to label positive those who truly have the
disease or ability of test to detect disease when the disease is
present
a. Sensitivity
b. Specificity
c. Accuracy
d. Positive Predictive Value

3. A screening test must ideally have

a. High degree of sensitivity and a low false positive error
b. A high degree of specificity and a low false negative
error
c. A high degree of sensitivity and a low false negative
error
d. A high degree of specificity and a low false positive
error

4. Which among the following describes confirmatory test?

a. Used for further analysis of a sample to confirm a
positive or sometimes, negative result
b. Used to detect early disease or risk factors for disease
in large numbers of apparently healthy individual
c. Used to rule in true hypotheses/diagnoses
d. A and C
e. All of the above

5. True Positive is:

a. Positive test result, no disease
b. Positive test result, has disease
c. Negative test result, has disease
d. Negative test result, no disease

AACDC

Transcribers: Garcia, Gesmundo, Gibe, Gonzales, Gonzalez

Editors: Armada
[EPI][Accuracy of Observations and Measurements] Page 5 of 5