SOME CONSIDERATION IN CONSTRUCTING TESTS

1. 2. 3. 4.

5.

PRACTICALitY RELIABILITY VALIDITY AUTHENTICITY WASHBACK

1) AN EFFECTIVE TEST IS PRACTICAL, IF:

It

is not Excessively Expensive It stays within appropriate time constraints It is relatively easy to administer, and It has a scoring/evaluation procedure that is specific and time-efficient

2) RELIABILITY
A Reliable

test is consistent and dependable

It has several fluctuations: o Student-Related Reliability o Rater Reliability o Test Administrator Reliability, and o Test Reliability

Student-Related Reliability

The most Common issues in this

categories

by physical or psychological factor such as temporary illness, fatigue, a bad day and anxiety. Also such factors as a test-takers test-wiseness or strategies for efficient test taking

caused

Rater Reliability Human error, subjectivity and bias may occur in the process of scoring. In rater reliability there are two categories: inter-rater reliability; it occurs when two or

more scores yield inconsistent scores of the same test

intra-rater reliability; is a common

accurance for classroom teachers because unclear scoring, fatigue, bias toward particular good and bad student.

Test Administrator Reliability

How the test administered and managed.

Test Reliability

Sometimes the test itself can cause measurement error Such as: the time limitations or ambiguous test items

3) Validity
This term refers to the extent to which a test measures what it says it measures. (Coomble and Hubbley, 2009,5)
According to McNamara(2000): it is the reltionhsip between evidence from the test performance and infernces about candidates capacity to perform in the criterion that are drwn from that evidence. In other words, test what you teach, how you teach it!

VALIDITY

CONTENT VALIDITY

CONSTRUCT VALIDITY

FACE VALIDITY

Content Validity

For classroom teashers it means that the test assesses the course content and outcomes using format familiar to the student.

Construct validity
Refers to the fit between the underlying theories and methedology of language learning and the type of assessment. Example: a communicative language learning approach must be matched by communication language testing

Face Validity

It means that the test looks as though it measures what it is supposed to measure
Face vallidity will likely be high if leaners encounter:

Face Validity

Criterion-related validiy

Consequential Validity

Criterion-related validiy:
The

Consequential Validity:

4)Authenticity

Bachman and Palmer (1996,23) in Brown (2004) define authenticity as:

the degree of correspondence of the characteristics of given language test task to the future of the target language task

5) Washback

Coombe nad Hubley (2009:8) Proposed proposed two other criteria before constructing test: Trasparency

and Security

Trasparency: the avalability of clear, accurate

Information to the student about testing

Constructing the test

Before constructing the test we must consider following process in developing a test: Planning Test Content and development Before the test Tes adminnistration After the test Reflect on the testing

Planning
a) Establish purpose of the test
Place student program Acheivement of course outcomes Diognosis of the strengths and areas improvements International benchmark b) Identify objectives Operationalizes outcomes c) Decide on cut-offs Grade, mastery d) Inventory course content and materials Consider approriate formats Establishe overall weighting e) Scheduling f) Write test specifications

Test Content and development

a) b) c) d) e)

Map the exam Decide on sections, formats, weighting Construct items according to test specifications Establish grading criteria Prepare an answer key Vett the exam Pilot the exam

Before the Test

Provide information to the students Coverage, weighting, formats, logistics b) Prepare the Students Student test-taking strategies Practice exam activities
a)

Test Administration
a). Decide on test conditions and procedures b) Organize equipment needed c) Establish makeup policy d) Inform student about availability of result

After the Test

a) Grade test Calibrate if more than one teacher involved Adjust answer key if needed b) Computer basic statistics c) Get result to students Provide feedback for remediation d) Conduct exam analysis Overal exam analysis Item and distractor analysis Error analysis e) Report on exam analysis Channel washback

Reflect on the testing process

a) b) c) d) e) f)

Learn from each exam Did it serve its purpose? What was the fit with curricular outcomes? Was it valid and reliable test? Was it part of the studentss learning experience? What future changes would you make?