Chapter Two

Measurement, Reliability &

Validity

“If you can’t measure it,

you can’t manage it.”
Bob Donath,
Consultant

© 2010 by Nelson Education Ltd. 1

 Chapter Learning
Objectives
After reading this chapter you should be able to:
• Understand the basic components that make up a traditional personnel
selection model;
• Know what a correlation coefficient is, along with a few other basic
statistical concepts used in personnel selection;
• Have a good understanding of the concepts of reliability and validity;
• Recognize the importance and necessity of establishing the reliability and
validity of measurements used in personnel selection;
• Identify common strategies that are used to provide evidence on the
reliability and validity of measurements used in personnel selection; and
• Appreciate the requirement for measurements used in personnel selection
to evaluate applicants fairly and in an unbiased fashion

© 2010 by Nelson Education Ltd. 2

 The Recruitment &
Selection Process
• Correlation:
– Measure of central tendency and variability
– Useful for bigger sets of observations
– Measures the relationship between two variables (Does not imply
causality!)
Type of investigation:
• Causal
– To establish cause-and-effect
Does job satisfaction (JS) cause performance?
• Correlational
– To identify associated variables
Are JS and performance (P) related?

– Measures direction of influence (+ or -)

– Correlation coefficient (r) ranges from -1 to +1
Correlation coefficient: a number or function indicating the degree of correlation between two
variables. It ranges between 1 for high positive correlation to -1 for high negative correlation,
with 0 indicating a purely random relationship.

© 2010 by Nelson Education Ltd. 3

Example of convergent validity within instrument.

© 2010 by Nelson Education Ltd. 4

© 2010 by Nelson Education Ltd. 5
 The Recruitment &
Selection Process

• Coefficient of Determination: r2
– Indication of the strength of a
relationship between two
variables is r squared.
– This value represents the
proportion of variability in one
variable that is associated with
variability in another .

© 2010 by Nelson Education Ltd. 6

 The Recruitment &
Selection Process
• Simple & Multiple Regression:
– The relationship between two variables can also be
expressed in terms of a straight line. This is a
regression line. (simple regression)
– Many practical situations involve more than two
variables (multiple regression)
– Regression is often used to predict performance

© 2010 by Nelson Education Ltd. 7

 Research on the “Big Five” Traits
Absenteeism
– Extraversion is positively related to absenteeism
– Conscientiousness is negatively related to absenteeism
– No differences found between racial groups, and thus less adverse impact when using for
selection purposes compared to cognitive ability tests

Agreeableness
– Needed for jobs requiring teamwork and good
interpersonal facilitation skills
– However, negatively related to career satisfaction
Extraversion
– Related to positive affectivity and interpersonal facilitation
– Positively related to salary levels, promotions, and career
satisfaction
Dark side to conscientiousness
– Not helpful for jobs that require creativity, innovation, spontaneity, or quick
decision making ("analysis paralysis")
Openness to experience
– Needed for jobs requiring innovation, creativity, and spontaneity
– May be predictive of cross-cultural sensitivity and success in international
assignments Emotional Stability (Neuroticism)
– Negatively related to salary levels obtained – Related to negative affectivity
– Cope poorly with stress
© 2010 by Nelson Education Ltd. – Negatively related to career
satisfaction
 Validity & Reliability

Only a
valid instrument

can generate
reliable information
© 2010 by Nelson Education Ltd.
© 2010 by Nelson Education Ltd. 10
 Validity

• Validity
– Legitimacy or correctness of the inferences that are
drawn from a set of measurements or other specified
procedures.
– The degree to which accumulated evidence and
theory support specific interpretations of test scores in
the context of the test’s proposed use.

© 2010 by Nelson Education Ltd. 11

 Validity
Evidence that the Instrument used to measure a concept does
indeed measure the intended concept
• Content: (Cover all dimensions and elements)

• Criterion: (Differentiate groups on a criterion)

Concurrent: Different should score differently:
Occurs when the criterion measures are obtained at the same time as the test scores. This indicates the
extent to which the test scores accurately estimate an individual’s current state with regards to the
criterion. E.g. IQ, EQ
Predictive : Predict future score:
Occurs when the criterion measures are obtained at a time after the test. Examples of test with
predictive validity are career or aptitude tests
• Construct: (Fit the theories)
Convergent: Different instrument highly correlate
Discriminant: Uncorrelated variables found
© 2010 by Nelson Education Ltd.
© 2010 by Nelson Education Ltd. 13
© 2010 by Nelson Education Ltd. 14
© 2010 by Nelson Education Ltd. 15
 How to Validate a Test

Step 1. Analyze the job

Step 2. Choose your tests

Step 3. Administer test

Step 4. Relate your test scores and criteria

Step5. Cross validate and revalidate

© 2010 by Nelson Education Ltd.

5
 Testing Guidelines

Basic guidelines for setting up a testing program include:

1. Use tests as supplements.

2. Validate the tests.
3. Analyze all your current hiring and promotion
standards.
4. Keep accurate records.
5. Begin your validation program now.
6. Use a certified psychologist.
7. Pay attention to test conditions.
© 2010 by Nelson Education Ltd.
6
 Equal Employment Opportunity
Aspects of Testing
In order to be in compliance with EEOC guidelines (with respect to testing),
you must be able to :

Prove that your tests were related to success or

failure on the job (validity).

Prove that your tests didn’t unfairly discriminate

against either minority or nonminority subgroups.

© 2010 by Nelson Education Ltd.

7
 The Recruitment &
Selection Process

• Reliability:
– The degree to which observed scores are free from
random measurement errors.
– Indication of the stability of a set of measurements
over repeated applications of the measurement
procedure

© 2010 by Nelson Education Ltd. 19

 Reliability
Evidence that the Instrument is stable and consistent
• Stability: (result remain the same over time)
Test-retest: Repetition to same respondents at two
different time
Parallel-form: Two comparable sets of measures
highly correlate. (order/ words change)

• Consistency: (Homogeneity of the items)

Inter item: degree of correlation among items
(Cronbach’s Alpha, 1946)
Split-half: degree of correlation among two halves
© 2010 by Nelson Education Ltd.
 Methods of Estimating
Reliability

• Measuring reliability
– Estimate the degree of variability that is caused by
measurement error
– Estimate can be obtained by using two different but
parallel measurements of the attributes
– Personal scores on both separate measures should be
relatively consistent

© 2010 by Nelson Education Ltd. 21

© 2010 by Nelson Education Ltd. 22
 The Selection Process

Three main reasons why careful selection is important:

Your own performance always depends in part on your

subordinates’. Employees with the right skills and
attributes will do a better job for you and the company.

It is very costly to recruit and hire employees. Search

fees, training, reference checking, moving expenses all
add up to many thousands of dollars.

There are many legal implications of incompetent

selection. Hiring workers with criminal backgrounds,
and who commit crimes in the course of their employ-
ment, may expose the employer to liability.
© 2010 by Nelson Education Ltd.

2
Basic Testing Concepts; Validity

Test validity answers the question, “Does this test measure

what it’s supposed to measure?”

Demonstrating criterion validity means demonstrating that

those who do well on the test also do well on the job, and
those who do poorly on the test do poorly on the job.

The content validity of a test is demonstrated by showing

that the test constitutes a fair sample of the content of the
job.

© 2010 by Nelson Education Ltd.

3
Basic Testing Concepts; Reliability

Reliability is a test’s second important character-

istic and refers to its consistency. It is “the con-
sistency of scores obtained by the same person
when retested with the identical tests or with an
equivalent form of a test.”

© 2010 by Nelson Education Ltd.

4
 Types of Tests

Tests of Cognitive Abilities

Intelligence Tests
Specific Cognitive Abilities

Tests of Motor and Physical Abilities

Measuring Personality and Interests

Achievement Tests

© 2010 by Nelson Education Ltd.

8
 Other Selection Techniques

Background Investigations and Reference

Checks

The Polygraph and Honesty Testing



Graphology


Physical Examination


Drug Screening


© 2010 by Nelson Education Ltd.

9
Detecting Dishonesty; Some
Guidelines

Ask blunt questions

Listen, rather than talk
Ask for a credit check
Check all references
Test for drugs
Conduct searches
© 2010 by Nelson Education Ltd.
10
Guidelines for Collecting Background
Information

1. Check all applicable state laws.

2. Check beyond applicable state laws to the impact of federal / other
equal employment laws.
3.Do not obtain information that will not be used.
4.Remember that using arrest information will be highly suspect.
5. Avoid blanket policies (such as “we have no one with a record of
worker’s compensation claims”).
6.Use information that is specific and job related.
7.Keep information confidential and up to date.
8. Never authorize an unreasonable investigation.

© 2010 by Nelson Education Ltd.

11
 Bias and Fairness

• Bias
– systematic errors in measurement or inferences made
from measurements
– related to different identifiable group membership
characteristics such as age, sex or race.

© 2010 by Nelson Education Ltd. 30

 Bias and Fairness

• Fairness
– the value judgments people make about the decisions
or outcomes that are based on measurements
– Based upon perceptions
– Cannot empirically or statistically be determined

© 2010 by Nelson Education Ltd. 31

 Summary

• One goal of personnel selection is to use

scientifically derived information to predict which
job applicants will do well in the job.
• The procedures used to select employees must meet
acceptable professional standards.

© 2010 by Nelson Education Ltd. 32