Professional Documents
Culture Documents
Based on the McMaster
Undergraduate MD Programme
[ objective
² use multiple choice questions
[ knowledge
² test what learners know
[ over time
² test repeatedly at regular intervals
[ curriculum-wide
² address end-of-programme learning objectives
|
[ to determine whether the learner is
progressing
² learning enough?
² retaining enough?
² timely feedback
[ reassure those who are progressing
[ alert those who are not
² do so early enough to facilitate effective
remediation
|
[ help the Programme (summative evaluation)
² provide defensible evidence to support
critical decisions pertaining to individuals
[ mandated remediation
[ pass / fail / conditional advancement
[ the emphasis..
² on formative aspects
[ minimize negative impact on learning
behaviours
² tutorial functioning
² self/peer-assessment
|
[ the item bank
² a sample of the knowledge that a good
student will likely encounter by the time
that student. .
[ graduates?
[ is six months / a year beyond graduation?
² don't guess
[ test your ability to self-assess
[ penalty for guessing (optional)
[ look at patterns..
² across time
[ week in programme (x of 138)
² across classes at week x
[ % attempted, % correct
² patterns are relatively stable across tests
and classes
² means at week x are relatively consistent
across tests and classes
² examinee performance is relatively
consistent across tests and classes
[ overall test reliability 0.6 - 0.7 (8 tests)
[ test-retest correlation 0.6 - 0.8 (2 tests)
|
Any questions?
è
1. Introduce progress testing
2. Using SIR for progress testing
² what's in an item?
² data management tasks
² managing dm tasks
[ software
[ databases and pql
² SIR \ valued features
² future enhancements
!"
[ the examinee sees..
nn. 2
Stem
!"#$ %
&
2 2
B '
C 2(
Options
D )
E *
!#
[ the data manager sees..
² stem and options (text)
and
² unique item identifier
² correct response code
² content codes (6 fields, 1, 2 or 3 sets)
² item performance data
[ stats on usage, power to discriminate
[ by test, class; across tests, classes
² and more..
[ date last used, don't use flag
$
1. store, retrieve, manipulate and print
large volumes of textual information
[ pre-test: test booklets
² 180 items, 21²22 pages/booklet
[ post-test: performance reports
² for examinees: 2 reports x 1²2 pages/rep
² for administrator: re. items, tests, classes and
examinees who are not progressing
[ accommodate special needs re.
² special characters ² Greek letters, math symbols
² page layout, fonts, typeface style
[ merge data into report templates
$ %
2. read examinees' responses
² 100-item optical mark response sheets
² tab-delimited ascii records
[ Mac: 2 sheets X approx. 280|420 examinees
Corel WordPerfect
² global variables
[ pass user settings to applications
² utilities
[ Data \ File Dump, File Input
[ PQL Procedures
² write csv-format records
² xtab tables, flexibility re. headers (columns)
² write SPSS system files
º
[ upgrade to SIR 2002 (from SIR 2000)
² update custom applications (to vb pql)
² add secondary indexes
[ examinees by name, current class
[ web access
² for examinees: performance reports
² method?
[ ColdFusion (CF ²² SQL ²² ODBC driver ²² SIR db)
[ CGI scripts
|
Any questions?