POLICY ANALYSIS

PROFESSIONAL JUDGEMENT


Joyce Thomas
To make grades more educative,
provincial ministries of education must
state clearly what they see as the
primary purpose of grades and then
ensure that their policies are aligned
with that purpose. When that happens
it will be possible to say that we have
learned from the history of grading and
that schools are focused on learning,
not grades.
(OConnor, 2010, p. 41)
Definitions of Professional
Judgement in Assessment

According to;
Scates (1943) - Teachers assessments
are based on interplay between
objectivity and judgement and respond
to criteria that differ from those of
scientific measurement.
(as cited by Allal, 2012, p. 21)


Definitions of Professional
Judgement in Assessment
Smith (2003) sufficiency of information, which implies
that the teacher seek a sufficient amount of relevant
information to be able to assign the grade that best
represents the students level of learning in the area
under consideration.
(as cited by Allal, 2012, p. 21)

Moss (2003) reconceptualization of validity, based on
hermeneutics and social cultural theory for
assessment. Including interpretation of qualitatively
different sources of information about student
learning.
(as cited by Allal, 2012, p. 21)

Definitions of Professional
Judgement in Assessment


Harlen (2005) Systemic review of the
dependability of teachers assessments
prioritising the validity component of
dependability over that of reliability.
(as cited by Allal, 2012, p. 21)

WHEN IS TEACHERS
JUDGEMENT PROFESSIONAL?

We propose to qualify teachers judgement
as professional when it takes into account
the resources and constraints of their work
setting and is informed by professional
knowledge acquired through experience and
through training. (Allal, 2012, p.20)

How We Become Experts:
Mastery, Expertise and
Excellence
Bereiter & Scardalmalia (Surpassing Ourselves, 1993)

the difference between experts and non-experts is not
that one does things well and the other does things badly.
Rather, the expert addresses problems whereas the
experienced non-expert carries out practiced routines. It
is only when the routines fail that the difference between
experts and non-experts becomes manifest(p. 11)
(as cited by Barber, 2014, ppt. slide 12)





(Barber, May 26, 2014, ppt. slide 12)
PROFESSIONAL JUDGEMENT
AS A COGNITIVE PROCESS
The findings of a case study by Allal
(2012) which investigated teachers
professional judgement in assessment,
showed that teachers judgements
about end-of-term grades involved two
main operations: (a) gathering
information from a variety of sources;
and (b) combining information in an
interpretative synthesis that sometimes
included but was not limited to an
arithmetic algorithm. (Allal, 2012, p.26)
PROFESSIONAL JUDGEMENT AS A
SOCIALLY SITUATED PRACTICE

When explaining their practices and
grading judgements, many teachers
referred to discussions in their school
about the importance of coherence in
communication concerning assessment
with students and their parents. This
concern often led to the elaboration of a
policy regarding summative assessment
which was followed on a school-wide
basis. (Allal, 2012, p. 28)

THE FAIR AND EQUITABLE ASSESSMENT
OF STUDENTS, REQUIRES TEACHERS
TO CONSIDER AVAILABLE EVIDENCE ON
A CASE BY CASE BASIS

This process of combining quantitative and
qualitative information entailed an
interpretative synthesis in which the teacher
decided how much importance would be
given to various types of information and
which pieces of information appeared to be
most relevant in the case of a given
student. (Allal, 2012, p.27)

PROFESSIONAL JUDGEMENT REQUIRES
THE USE OF ALL AVAILABLE
INFORMATION
From the teachers viewpoint, you have to
use all the information you have about a
student to formulate the judgement that is
the most appropriate for that individual.
Although you may lack equivalent
information about other students, it does not
diminish your responsibility to use
judiciously the information you have. (Allal,
2012, p. 32)

What is the point of Professional
Judgement?
In a study by DAgostino and Welsh (2007) they found that
teachers who used high-quality grading procedures (good
judgement) would be more accurate in their appraisal of
students against state standards than those who did not.
(as cited by Brookhart, 2012, p. 77)

High-quality grading procedures includes the following
considerations: the use of quantitative and qualitative
assessments; most consistent and most recent
demonstration of achievement; the use of transparent
assessment practices that are fair, equitable, criterion
referenced; does not include the use of zeroes or late
penalties; includes collaboratively developed rubrics and
success criteria; provides for re-writes, re-test, descriptive
feedback and scaffolding.
What is the point of Professional
Judgement?
The point is a final grade:
1. It is not the average of the students
evaluations.
2. It is not the zeroes added, or the marks
deducted.
3. It is the use of Professional Judgement
through the Triangulation of the DATA
(product, conversation and observation).

Doug Reeves: Toxic Grading Practices
https://www.youtube.com/watch?v=jduiAnm-
O3w
Growing Success Document
The Ontario Ministry of Education: Growing
Success 2010 document regarding evaluation and
the teachers use of best professional judgement,
in the determination of a final grade, clearly states
that teachers are to evaluate the student in terms of
achievement of overall curriculum expectations.

Teachers will weigh all evidence of student
achievement in light of these considerations and will
use their professional judgement to determine the
students report card grade. The report card grade
represents a students achievement of overall
curriculum expectations, as demonstrated to that
point in time. (Growing Success, 2010, p.39)


DIAGNOSTIC ASSESSMENT IN SUPPORT OF
STUDENT LEARNING Ont. Ministry of Education
Policy Program Memorandum 155

This memorandum also outlines how teachers shall
use their professional judgement to determine:
which assessment and/or evaluation tool(s) from the
boards list of pre-approved assessment tools is
applicable;
for which student(s); and
the frequency and timing of the use of the tool.
(Ministry of Education, PPM 155)


DIAGNOSTIC ASSESSMENT IN SUPPORT OF
STUDENT LEARNING Ont. Ministry of Education Policy
Program Memorandum 155

In order to inform their instruction,
teachers must utilize diagnostic
assessment during the school year.
A teachers professional judgement is
the cornerstone of assessment and
evaluation. (Ministry of Education,
PPM 155)
Collective Responsibility and Professional Judgement

Teachers, principals, and school board staff
share a collective responsibility and
accountability for student achievement and, in
their respective roles, exercise their
professional judgement as defined in Growing
Success:
Judgement that is informed by professional
knowledge of curriculum expectations, context,
evidence of learning, methods of instruction and
assessment, and the criteria and standards that
indicate success in student learning. In professional
practice, judgement involves a purposeful and
systematic thinking process that evolves in terms of
accuracy and insight with ongoing reflection and
self-correction. (p. 152) (as cited by Ministry of Edu.
PPM 155)
District School Board Policy
5124 Regulation: Standards For Quality In The
Assessment, Evaluation And Reporting Of Student
Achievement
3.4 Assessment, Evaluation and Reporting is
Based on Individual Achievement

Even where a cooperative or independent learning
strategy is used, assessment will be based on
individual achievement. A group mark may be
assigned where the merit of the final product must be
considered in its entirety, providing that this was
communicated clearly to students in advance. (XDSB,
2013-2014, p. 2)



District School Board Policy

3.5 Communication of Feedback Will Be
Timely and Understandable to the
Stakeholders

Student achievement is typically reported using
report cards. Teachers should also use
communication methods such as portfolios,
conferencing, narrative or anecdotal reports,
checklists, rating scales, rubrics and test
scores, especially for communicating formative
evaluations of student achievement.
(XDSB, 2013-2014, p. 3)


Second District Board Policy Reviewed
Teachers have a leading role to play in the implementation of
the seven fundamental principles. On a daily and hourly basis,
teachers make professional judgements that ensure effective
implementation of these principles, making decisions with
respect to individual students and groups of students that have
profound implications for them. How students feel about
themselves as learners and whether they enjoy learning and
strive for excellence are closely related to their teachers
professional skills both in differentiating instruction and
assessment and in helping students understand how they can
improve.

Teachers create environments in which all students feel valued
and confident and have the courage to take risks and make
mistakes. In their important professional role, teachers show
students that they care about them, and model a love of learning
that can deeply influence their lives. Teachers professional
judgements are at the heart of effective assessment,
evaluation, and reporting of student achievement.
(XDSB, 2010, p. 3)
Second District Board Policy
Reviewed
PROFESSIONAL JUDGEMENT
Professional Judgement is informed by
professional knowledge of curriculum
expectations, context, evidence of learning,
methods of instruction and assessment, and the
criteria and standards that indicate success in
student learning.
In professional practice, judgement involves a
purposeful and systematic thinking process that
evolves in terms of accuracy and insight with
ongoing reflection and self-correction.
(XDSB, 2010, p. 6)
Second District Board Policy
Reviewed
3. EVALUATION
3.1 Assessment of Learning throughout the Course
For students achieving below 50%, teachers will work
in collaboration with the Student Success team, and
use their professional judgement to determine
grades for report cards.
(XDSB, 2010, p. 13)


Second District Board Policy
Reviewed
4. ABSENCE, LATES AND MISSED ASSIGNMENTS
4.1 Providing alternative assignments or tests/exams where, in the
teachers professional judgement, it is reasonable and appropriate to do so;
and deducting marks for late assignments, up to and including the full value
of the assignment. This strategy is to be used only after other options
have been exhausted. Teachers must ensure that mark deductions
will not result in a final grade that misrepresents the students actual
level of achievement.
(XDSB, 2010, p. 15)

4.4 If, in the teachers professional judgement, however, the student has
not demonstrated achievement of expectations of the missed evaluations in
other evaluations or in another context and/or the student does not have a
valid reason for the missed evaluation(s), the teacher may determine that
insufficient evidence of achievement has been provided by the student to
make an accurate and valid evaluation of student performance. The
teacher will consider the students most consistent overall level of
achievement on completed evaluations but will use professional judgement
to determine the mark. (XDSB, 2010, p.16)
Second District Board Policy
Reviewed
5. REPORTING
When completing report cards, teachers will review evaluation data recorded and will apply
professional judgement in determining whether the student has provided sufficient evidence
to
make a valid judgement about the students achievement of the expectations addressed
throughout the reporting period. Based on the extent of the evidence provided by the
student on
evaluation tasks and the teachers judgement as to the degree to which the student has
demonstrated achievement of the course expectations, teachers will assign a mark to
represent
the students most consistent overall level of achievement. This mark is simply a symbol to
represent the level of achievement demonstrated by the student. It is not an accounting of
right
and wrong answers.
The mark for each students term work should reflect the students most consistent level of
achievement throughout the course, although special consideration should be given to more
recent evidence of achievement, when a developing skill is being assessed. When discrete
knowledge is being assessed it is understood that most recent may not apply (e.g., a
biology unit
in science may have an evaluation early in the semester and biology will not be covered
again until
the final evaluation in a course; a mathematics strand is completed in the first reporting
period).
Most consistent is understood as the students typical demonstrated level of achievement.
(XDSB, 2010, p. 17)
Second District Board Policy
Reviewed
5.6 (e)
All mark reporting should reflect the
professional judgement of the teacher. Most
recent evidence of achievement should be
used when looking at growth over time.
Teachers should never only use the number
calculated by a computer program to assign
a grade for a report card. Teachers using
software programs should reflect on the
calculation and use professional judgement to
determine whether or not the calculated grade
best fits the students achievement of the
expectations in the subject. (XDSB, 2010, p.
17)

Second District Board Policy
Reviewed
Determining Final Grade for the Provincial
Report Card
5.10 If a student misses or does not complete one or more
components of the final evaluation, a zero may be assigned to
the missing components and used in the determination of the
final grade.

However, in rare cases where there is an extenuating
circumstance for the missed evaluation, the teacher, in
consultation with the school administrator, may use professional
judgement to determine whether it is appropriate to provide
either an opportunity for an alternative evaluation or to adjust
the grade with due consideration given to the circumstance for
the missed final evaluation. If there is more than one final
evaluation activity, teachers will assign a percentage grade for
each of the tasks based on the determined level of
achievement. Teachers will then weight appropriately the tasks
to determine the thirty percent portion of the mathematical
calculation of the final grade. (XDSB, 2010, p. 19)
My Research in reviewing two Boards Policies regarding
Assessment and Evaluation, specifically in regards to
Grading Practises
Professional Judgement, is not
referenced directly in the first school
District Board policy, however,
referenced numerous times in the
second, yet there is still no clear
definition of what teachers
professional judgement entails!


Teacher judgement in classroom
assessment is obviously subject to
error and bias, as is judgement in all
areas of professional expertise
(medical, financial, etc.) (Allal, 2012,
p. 21).

Policy into Practice
At the school level:
Teachers will benefit from leadership by
the principal to ensure that there is a
common understanding among all staff
about the process for determining the
final grade. The principal will work with
teachers to ensure common and
equitable grading practices that follow
ministry policy and board guidelines.
(Growing Success, 2010, p. 39)

What does this mean?
At the school level, the Principal works within the context of the
Board policies on assessment and evaluation, in order to set
common assessment and evaluation policies and practices within
the school.
The issue is a lack of consistency within a given School Boards
schools with respect to assessment and evaluation polices and
practices. For example: late marks, the use of zeroes, triangulation
(produce, conversion and observation) of data, plagiarism, and
especially professional judgement, when it comes to determining a
FINAL GRADE.


WHEN IS TEACHERS
JUDGEMENT PROFESSIONAL?

We propose to qualify teachers judgement
as professional when it takes into account
the resources and constraints of their work
setting and is informed by professional
knowledge acquired through experience and
through training. (Allal, 2012, p.20)
Professional Judgement and
Professional Learning Communities
does this happen?
teachers professional judgements are at the heart of
effective assessment, evaluation, and reporting of
student achievement (Growing Success, 2010, p. 8).
In addition, successful implementation of policy
depends on the professional judgement of educators at
all levels, as well as on educators ability to work
together on the continuing efforts of strong and
energized professional learning communities to clarify
and share their understanding of policy and to develop
and share effective implementation practices, on
creative and judicious differentiation in instruction and
assessment to meet the needs of all students, and on
strong and committed leadership from school and
system leaders, who coordinate, support, and guide the
work of teachers(Growing Success, 2010, p. 2).
(as cited in Ministry of Education, PPM 155)




How often do we train staff in developing their PROFESSIONAL
JUDGEMENT to align with GRADING PRACTISES?

(Project Evidence. Retrieved from http://teacherevidence.net/professional-learning/)



The Problem in Finding a
Solution
Principals are left on their own to determine
where their staffs Professional Judgement
comes in effect regarding grading practices.
The Growing Success 2010 document is
vague at best in providing standardised
assessment practices concerning teacher
professional judgement.
Thus the danger is that in the absence of
specific Board policy to direct the Principals
leadership in the fair and equitable
assessment and evaluation practices of
teachers, teachers are left to interpret and
apply professional judgement which leads to
inconsistency and inequality.
Future Research..
1. What are teachers thoughts as they
consider their judgments of student
achievements?
2. What characterises those teachers
whose judgement of their students
achievements are accurate?
(DAgostino and Welsh, 2007)
BEST PRACTICE IN THE USE OF PROFESSIONAL JUDGEMENT BY
SECONDARY SCHOOL TEACHERS IN THE DETERMINATION OF
COURSE FINAL GRADE

QUESTIONS:
1. Given that professional judgement is founded on subject expertise and experience in
teaching, what are the implications for school level: (a) policy development, and (b) staff
professional development?
2. What organizational structures are necessary to prevent the determination of final grades
through Best Professional Judgement that is conducted in isolation which can lead to
invalid and unreliable results?
3. The final grade represents a students achievement of the course Big Ideas, as
represented by the overall expectations for the course. What grade should a student
receive who achieved Level 1 (50-60%) in the first half of the course and Level 3 (70-80%)
in the second half including summative tasks and exam - of the course being graded?
4. Why does a zero on a 100 point scale, have a disproportionate effect on the final grade, as
compared to a zero on a 4 point scale? Part two of this question, what is the difference
between a mark of 45 and a mark of 15 on a 100 point scale, given that the goal of teaching
is student learning and the development of thinking skills? (i.e. -Blooms)
5. As the educational leader in your school, rank the following in order of priority:
A. A common language linked to shared beliefs.
B. Recording evidence of learning through student work, teacher observations, and student-
teacher conferences.
C. The use of common assessment tasks.
D. Teacher conferences on the creation and evaluation of assessment tasks.
E. The use of best professional judgement in determining a final course grade.
F. The use of criterion referenced (rubrics) based assessment and exemplars.

Works Cited
Allal, L. (2013). Teachers professional judgment in assessment: a cognitive act and a socially
situated practice. Assessment in Education: Principles, Policy & Practice, 20(1), 20-34. doi:
10.1080/0969594X.2012.736364
Barber, W. (2014, May 26). Re: How We Become Experts: Mastery, Expertise and Excellence
[Online forum lecture]. Retrieved from
https://uoit.blackboard.com/webapps/portal/frameset.jsp?tab_group=courses&url=%2Fwebapp
s%2Fblackboard%2Fexecute%2Fcontent%2Ffile%3Fcmd%3Dview%26content_id%3D_50919
5_1%26course_id%3D_16097_1%26framesetWrapped%3Dtrue
Bogush, P (Producer). (2010). Toxic Grading Practices-Douglas Reeves [YouTube]. Available from
https://www.youtube.com/watch?v=jduiAnm-O3w
Brookhart, S. (2012) The use of teacher judgement for summative assessment in the USA.
Assessment in Education: Principles, Policy & Practice, 20(1), 69-90. doi:
10.1080/0969594X.2012.703170
Durham District School Board. (2013-2014). 5124 - Regulation Standards For Quality In the
Assessment, Evaluation And Reporting Of Student Achievement. Policies and Procedures-
Durham District School Board. Retrieved from
https://www.intranet.durham.edu.on.ca/Applications/DDSBPPI.nsf/(webprint)?openagent&cate
g=Regulation:%20Students
Peterborough Victoria Northumberland and Clarington Catholic District School Board. (2010).
Assessment, Evaluation & Reporting Secondary Handbook: Grade 9 to Grade 12.
Peterborough Victoria Northumberland and Clarington Catholic District School Board.
Retrieved from http://www.pvnccdsb.on.ca/en/parents/resources/AERSecondaryHandbook.pdf
OConnor, K. (2010). Grades: When, Why, What Impact, and How? Education Canada, 50(2), 38-
41. Retrieved from http://www.cea-ace.ca/sites/default/files/EdCan-2010-v50-n2-O'Connor.pdf
Ontario Ministry of Education. (2010). Growing Success: Assessment, Evaluation, and Reporting in
Ontario Schools. First Edition, Covering Grades 1 to 12. Ontario: Queens Printer.
Ontario Ministry of Education. (2013). Diagnostic Assessment in Support of Student Learning.
Policy/Program Memorandum No. 155. Retrieved from
http://www.edu.gov.on.ca/extra/eng/ppm/ppm155.pdf