FRIT 7237 Program Evaluation Final Project Tim Gough, Julian Jernigan, Britton Spivey

Evaluation Topic

Our groups evaluation will be based in Savannah Chathams usage of the TKES (Teacher Keys Evaluation System) as an instrument for teacher review. Currently, Administrators must make several conferences, walkthroughs and observations for each and every teacher per academic year. On a standard campus, that may include six to eight visits, with duration varying from twenty minutes to one hour. Most schools employ thirty to sixty teachers The evaluation will focus not only on the improbable amount of time that the program needs to be completed school wide, but will also assess all components of the program including software and online functionality of completely a web based tool.

Question: Does the TKES instrument used for teacher evaluation, place achievable goals, and reasonable timelines on administrators to completely and competently track their facultys performance?

Check One For Each Item

Yes Step 1 Is there a contractual requirement to evaluate? (If yes, initiate the evaluation. If no, proceed to step 2.)

No

No, currently there is no contractual requirement for Savannah Chathams program evaluation. Move to step 2. Does the object of the evaluation have enough impact or importance to warrant formal evaluation? (If yes, proceed to step 3. If no, discontinue evaluation)

Step 2

The core of education is teaching and learning, and the teaching-learning connection works best when we have effective teachers working with every student every day. If teacher quality is the pillar of the success of education, then a robust teacher evaluation system should be in place. Is there sufficient consensus among stakeholders on the model for the program? Its goals and objectives? (If yes, proceed to step 4. If no, consider a needs assessment study.)

Step 3

Yes, a robust evaluation system will provide quality data that will benefit all stakeholders.

Data will include information on assess performance of educators for continuing contract and promotion decisions, determine recognition and awards for meritorious contributions, assist faculty and administrators in identifying strengths and needs for improvement, plan meaningful staff development activities, develop remediation goals and activities, when necessary, support fair, valid, and legal decisions for termination. Has the program been implemented before and data is available for stakeholders use? (If yes, proceed to step 5. If no, consider doing more research for additional data)

Step 4

Twenty-six Race To The Top (RT3) districts participated in the spring 2012 pilot of the Teacher Keys Effectiveness System (TKES) and Leader Keys Effectiveness System (LKES). The purpose of the pilot was to (1) improve administration procedures, training, and educator familiarity with the system, and (2) collect preliminary data that can inform instrument revision and validation plans prior to full operational implementation. Information is available at http://www.gadoe.org/School-Improvement/Teacher-and-Leader Effectiveness/Documents/Pilot%20Report_Overview%20and%20Report%20Combined%20110-13.pdf Are the objectives clear, realistic, and commonly understood among all stakeholders? (If yes proceed to step 6 if not consider a revision plan)

Step 5

Yes, Teacher Keys Effectiveness System (TKES), is a common evaluation system that will allow the state to ensure consistency and comparability across districts, based on a common definition of teacher effectiveness. If TKES is not an effective evaluation program, then the

program should no longer be used to evaluate teachers effectiveness. Is training available to administrators and teachers. Step 6 (If yes, proceed to step 7. If no, consider developing proper training before evaluating the program.)

Yes, Georgia has implement the Leader Keys Effectiveness System (LKES). The goal of Georgias Leader Keys Effectiveness System is to provide leaders with meaningful feedback and to support the evaluation process. LKES offers clear and precise indicators and resources to leaders throughout the process. Teacher will be given access to information in both online and hard copy form. Teachers will also participate in training provided by the school system. Are the performance indicators monitored adequately? (If yes proceed to step 8. If no, consider more training for evaluators)

Step 7

Teachers will be rated on the performance standards using performance appraisal rubrics. The performance rubric is a behavioral summary scale that guides evaluators in assessing how well a standard is performed. Teachers will receive a minimum of 6 visits that range from 20-60 minutes. Teachers will also provide other documentation to provide evidence of performance standards. Does the timing of the evaluation fit into the programmed cycle? (If yes, proceed to step 9. If no, consider revising time frames to fit the school calendar year)

Step 8

Yes, each administrator has one academic school year to complete all evaluations.

Responsibility is divided up among the administration team. Are the stakeholders in a position to use the information productively? (If yes, proceed to step 10; if no, discontinue or focus on other stakeholders who can use the information to make decisions or take action).

Step 9

Yes, school board members, superintendants, administrators, parents, teachers, and students will contribute to the data and use it appropriately for their needs. Do all components of the program fit the objective goals? (If yes, proceed to step 11. If no, discontinue the evaluation.)

Step 10

Yes, the program is made up of two major components: Teacher Assessment on Performance Standards and Measures of Student Growth and Achievement. Together these components provide a stronger assessment of teacher effectiveness than each of the measures alone and than existing measures. Should the evaluation take place? (If yes, conduct evaluation. If no, discontinue evaluation)

Step 11

Yes, all answered questions indicate that an evaluation is conducive and will provide important feedback to all stakeholders.

Question: Does the TKES instrument used for teacher evaluation, place achievable goals, and reasonable timelines on administrators to completely and competently track their facultys performance?

Evaluation Question Does the TKES process allow administrators to monitor their facultys performance?

Criteria Responses to survey and interviews completed by currently employed administrators and teachers.

Standard 75% of teachers and administrators agree the TKES process is effective in monitoring facultys performance.

Do stakeholders agree the TKES instrument includes standards necessary to effectively evaluate teachers?

Responses to survey and 75% of the schools interviews completed by current stakeholders agree members of the school the TKES instrument community including currently is an effective tool to employed administrators, evaluate teachers. teachers, parents and community members.

Can all components of the Responses to interviews and data 90% of school TKES evaluation be collected from TKES records of systems complete all completed within the evaluation submissions. components of the given timeframe? TKES evaluation within the given timeframe.

Question Does the TKES process allow administrators to monitor their facultys performance?

Information Required School system currently employed in. Years of in-field experience. Primary role in school system. Level of experience with TKES process. School system currently employed in. Primary role in school community. Level of experience with TKES standards.

Design

Information Source Currently employed teachers and administrators.

Cross-Sectional Time Series Case Study

Method for Collecting Information Survey(s). Interviews.

Do stakeholders agree the TKES instrument includes standards necessary to effectively evaluate teachers?

Cross-Sectional Time Series Case Study

Currently employed teachers, administrators, and parents of school system.

Survey(s). Interviews. Records of TKES data.

Can all components of the TKES evaluation be completed within the given timeframe?

School system currently employed in. Years of in-field experience. Level of experience with TKES standards.

Cross-Sectional Time-Series Case Study

Currently employed administrators.

TKES records for submissions of summative assessments (completed evaluations). Interviews.

Question Does the TKES process allow administrators to monitor their facultys performance?

Sampling

Survey to all teachers and administrators employed in a school system.

Information Collection Procedures Quantitative and qualitative data will be collected through survey and interviews.

Schedule

Analysis Procedures Use results from survey and feedback from interviews to determine the validity of both the quantity and the quality of completed teacher evaluations, to ensure a true measure of teacher performance.

Survey- Late May(Last two weeks of school year) Data formatted into report/statistics by Early June during or after post planning. Released to Teachers and Administrators during pre planning Survey- June/July Data formatted into report/statistics by July Released to Stakeholders at first PTA/PTSA

Do stakeholders agree the TKES instrument includes standards necessary to effectively evaluate teachers?

Survey to all teachers, administrators, and parents in school system.

Quantitative and qualitative data will be collected through survey, interviews, and TKES data.

Use results from the survey and feedback from interviews to determine if further professional development is required to train evaluators or explain TKES standards to school communities. Analyze teachers effectiveness levels (exemplary, proficient, needs development, ineffective) and

monitor if percentage of ineffective teachers decreases over time.

Can all components of the TKES evaluation be completed within the given timeframe?

Survey of all administrators employed in a school system.

Quantitative and qualitative data will be compiled through interviews and TKES record of evaluation submissions.

Detailed statistics from TKES data. Identify ratio of completed/incomplete Data formatted into evaluations. report/statistics by Use results to Early June during or determine which after post planning. school systems have met standards and Released to conduct interviews Administrators/School with evaluators. improvement team Use feedback to First of July determine plan of action for school systems not meeting timeframe requirements.

Survey- Late May(Last two weeks of school year)

Question Does the TKES process allow administrators to monitor their facultys performance?

Interpretation Procedures Did 75% of teachers and administrators agree the TKES process is effective in monitoring facultys performance? For school systems in disagreement of the process, what elements are missing in order to implement the TKES process adequately?

Audience(s) Teachers Administrators

Content To determine if teachers and administrators feel they can/are accurately portraying/portrayed their competency levels throughout the TKES observation/ walkthrough/ conference process. To determine an action plan for noncompliant schools

Format Hard copies of data report showing compiled results of all surveyed schools. A digital presentation(Prezi, Power Point etc) of results to distribute to administrators to disseminate to site specific employees Professional development on each of the ten standards included in the TKES Instrument. Hard copies of data report showing compiled results of stakeholder approval rating on a per standard basis. A digital presentation(Prezi, Power Point etc) of results to distribute to administrators to disseminate to site

Schedule Meeting to discuss results end of July. Report release late July prior to school improvement team meeting in Late July.

Do stakeholders agree the TKES instrument includes standards necessary to effectively evaluate teachers?

Did 75% of the school community agree the TKES instrument is an effective tool to evaluate teachers? Is there a correlation between documentation and teacher performance level? For systems with

Teachers Administrators Parents

To determine if the TKES standards effectively track/portray teacher performance levels. Do teachers that are evaluated with the TKES instrument have higher student proficiency levels each school year

Meeting to discuss results end of July. Report release late July prior to school improvement team meeting in Late July.

higher percentages of ineffective teachers, what interventions can be implemented to increase teacher effectiveness levels?

than those that are not evaluated with the TKES instrument? To determine an action plan for schools with high percentage of ineffective teachers Administrators

specific employees PDP for systems with high percentages if non proficient teachers.

Can all components of the TKES evaluation be completed within the given timeframe?

Did 90% of school systems complete all components of the TKES evaluation within the given timeframe? For those systems who did not meet the mandated timeframe, what resources and tools can be provided to ensure the timeline is followed in the future?

To determine the Hard copies of data Meeting to discuss actions and report showing results in June. planning necessary compiled results of Report release to comprehensively administrator beginning of July evaluate all teachers compliance on a per prior to annual on a given site on an site basis. pre-school annual basis. A digital administrators presentation(Prezi, conference. To determine a Power Point etc) of Professional results to distribute Development Plan to administrators. for administrators Professional that failed to meet Development Plan the mandated for TKES timeline timeframe compliance.

Question

Audience for Report Teachers Administrators

Report Content

Report Format

Reporting Schedule

Does the TKES process allow administrators to monitor their facultys performance?

Completed Teachers evaluations, conferences, summative administrators reports on teachers competency levels based on the TKES criteria. Teachers performance statistics based on holistic and analytical measures.

Written report, with oral briefing, including an executive summary.

Meeting to discuss results end of July. Report release late July prior to school improvement team meeting in Late July.

Context for Presenting Report Describe frequencies, percentages or averages found in quantitative data. Summarize patterns detected in observational records or clusters of answers in interview responses found in qualitative data.

Do stakeholders agree the TKES instrument includes standards necessary to effectively evaluate teachers?

Teachers Administrators Parents

Comparative data of student proficiency levels of TKES evaluated teachers vs. non TKES evaluated Teachers .

Presentation on findings using presentation software such as Prezi, PowerPoint etc.

Meeting to discuss results end of July. Report release late July prior to school improvement team meeting in Late July.

Presentation at parent involvement and staff development meeting of informational findings.

Can all components of the TKES evaluation be completed within the given timeframe?

Administrators

Problematic findings using teacher evaluation data.

Date and times of each required evaluation on each staff member. Report created using MS Word or Google Docs.

Meeting to discuss results in June. Report release beginning of July prior to annual preschool administrators conference.

Briefing and discussion of report during principals meeting.