An Accountability & Assessment Brief: 2011 CRCT Scale Scores and Cut Scores

2011 Scale Scores and Cut Scores for the CriterionReferenced Competency Tests

Scale Score Ranges

Scores on all reports are expressed as scale scores. With the phasedin implementation of the Georgia Performance Standards (GPS) complete, the Criterion-Referenced Competency Tests (CRCT) have now completed their transition to the GPS scoring system. The scale score reported for each content area is derived by converting the number of correct responses on the test (the raw score) to the CRCT scale. Since the scale scores are equivalent across test forms within the same content area and grade, students obtaining the same score have demonstrated the same level of performance with respect to the Georgia GPS. Scale scores are comparable across all test forms and administrations for the same content/grade. A scale score of 820 from one administration, for example, indicates the same examinee ability as a score of 820 from any other administration. The content and associated performance standards differ for each grade and content area assessment. Therefore, scale scores are not comparable across content areas or grade levels. GPSbased scale scores are NOT comparable to QCCbased scale scores from previous years since the content and test standards have changed. Scores on GPSbased CRCT assessments are structured to range from 650 to 910 or above. Variations in test characteristics and student performance result in different lower and upper limits (also known as Lowest and Highest Obtainable Scale Scores) for each grade and content area. However, scores above 900 on GPSbased CRCT generally indicate exceptional performance. The chart below shows the Lowest Obtainable Scale Score (LOSS) and the Highest Obtainable Scale Score (HOSS) for each grade/content area assessment of the CRCT. English/Language Arts LOSS/HOSS 650/910 650/910 650/930 650/930 650/930 650/930 650/930 650/950

Grade

Reading LOSS/HOSS

Mathematics LOSS/HOSS 650/930 650/930 650/990 650/990 650/990 650/950 650/950 650/990

Science LOSS/HOSS

Social Studies LOSS/HOSS

1 2 3 4 5 6 7 8

650/920 650/920 650/920 650/920 650/920 650/920 650/920 650/920

650/990 650/990 650/990 650/960 650/960 650/960

650/950 650/950 650/950 650/980 650/980 650/990

Georgia Department of Education Dr. John D. Barge, State School Superintendent Copyright 2011 Page 1 of 3

An Accountability & Assessment Brief: 2011 CRCT Scale Scores and Cut Scores Scale Scores and Performance Levels
A process known as standard setting helps to define points along the scale score range and gives additional meaning to student performance. New CRCT test performance standards were set as each grade/content area transitioned to GPS. Georgia educators and stakeholders from around the state participated in the standard setting process. For the CRCT three performance levels have been defined: Does Not Meet the Standard, Meets the Standard, and Exceeds the Standard. Scores that are at or above 850 on GPSbased CRCT, indicate a level of performance that Exceeds the Standard set for the test. Scores from 800849 for GPSbased CRCT, indicate a level of performance that Meets the Standard set for the test. Scores below 800 on GPSbased CRCT, indicate a level of performance that Does Not Meet the Standard set for the test (i.e., the states minimum level of proficiency). Test developers build parallel forms for each grade/content assessment for the CRCT. Even after applying the best psychometric principles in test construction, forms may differ slightly in difficulty from one year to the next. For this reason, tests are equated and results are reported in scale scores, not raw scores. Scale scores are used to allow comparison across years.

How Cut Points Define Performance Levels

During the standard setting process, a panel of Georgia educators and other stakeholders is selected for their expertise in each grade and content combination. The participants identify how many test questions a student must answer correctly to be classified as just meeting standard or just exceeding standard. The raw scores that define one performance level from another are called cut points. The cut points in raw scores are based on the initial administration for that test. When the performance standards are set for a test, the student ability (known as theta among psychometricians) at these cut points is established. These theta estimates are utilized to set the raw score points for each administration. Subsequently, the raw scores are converted to the scale score metric. This is the behind the scenes process that allows student performance from one year to another to be comparable. The thetas for the meets and exceeds performance levels remain constant across years and correspond to the established scaled cut scores. However, the corresponding raw scores for meets and exceeds may vary slightly because tests may differ slightly in difficulty from year to year. The chart below shows the meets and exceeds scale cut scores for the 2011 CRCT and the corresponding raw score for each grade/content area test. English/Language Arts 50 Scale Cut/ Raw Cut 800/24 850/42 800/27 850/44 800/24 850/43 Social Studies 60 Scale Cut/ Raw Cut 800/30 850/49 800/29 850/49 800/30 850/49

Content Area Grade Number of Items Possible Performance Level Meets Exceeds Meets Exceeds Meets Exceeds

Reading 40 Scale Cut/ Raw Cut 800/18 850/33 800/21 850/35 800/18 850/33

Mathematics 60 Scale Cut/ Raw Cut 800/34 850/48 800/35 850/50 800/31 850/47

Science 60 Scale Cut/ Raw Cut 800/31 850/47 800/32 850/48 800/33 850/47

3 4 5

Georgia Department of Education Dr. John D. Barge, State School Superintendent Copyright 2011 Page 2 of 3

An Accountability & Assessment Brief: 2011 CRCT Scale Scores and Cut Scores
English/Language Arts 50 Scale Cut/ Raw Cut 800/25 850/44 800/26 850/43 800/25 850/42 Social Studies 60 Scale Cut/ Raw Cut 800/31 850/47 800/32 850/47 800/32 850/48

Content Area Grade Number of Items Possible Performance Level Meets Exceeds Meets Exceeds Meets Exceeds

Reading 40 Scale Cut/ Raw Cut 800/18 850/34 800/19 850/35 800/19 850/35

Mathematics 60 Scale Cut/ Raw Cut 800/29 850/50 800/28 850/49 800/32 850/48

Science 60 Scale Cut/ Raw Cut 800/29 850/49 800/28 850/49 800/32 850/50

6 7 8

Caution is always advised when looking at the raw scores at the cut points. The number of correct responses (i.e., raw cut score) to obtain a performance level will vary slightly from year to year because tests may vary in difficulty from year to year. Scale scores are appropriate for making longitudinal comparisons within a content area as long as the tests reflect the same curriculum. For more information on interpreting scores, find the 2011 CRCT Score Interpretation Guide on the Georgia Department of Education website: http://www.gadoe.org/ci_testing.aspx?PageReq=CI_TESTING_CRCT

This brief is produced by the Assessment Research and Development Division of the Georgia Department of Education. Questions should be directed to Assessment and Accountability at 404-656-2668.

Georgia Department of Education Dr. John D. Barge, State School Superintendent Copyright 2011 Page 3 of 3