Executive Summary The 20102011 academic year was remarkable in numerous ways.

It would be inappropriate to try and list them all. Nonetheless, some highlights make the point. For instance, the historic vote by the citizens of Memphis to surrender the MCS Charter captured our attention for months and the aftermath continues. Still, Booker T. Washington remained focused, got its work done, and became the heartthrob of MCS when its work captured the attention and presence of President Barack Obamaa historic moment of a different kind and magnitude. Our graduation rate for the 20092010 academic year was 70.5%, the best of all urban districts in the United States; the data of the Memphis Police Department informs that our schools are safe; our MCS Prep Schools graduated 582 students whose divergence had them destined for a different reality; White Station Middle had students and their school acclaimed for their prowess in science; MCS got As for writing in grades 5, 8 and 11, a continuation of excellence in this content area; 172 of our teachers were selected by their peers to be Prestige Award winners; a delegation of our student envoys represented MCS at an invitation-only youth summit in Washington, D.C.; over 10,000 citizens participated in ThinkShow!; and ArtsFest slogged through rain and mud with every arts organization in Memphis to break new ground; over 1,000 MCS student earned college credit through our Dual Enrollment Programthe largest of its kind in the country; and many of our school had data that revealed significant growth in student performance at multiple grade levels. The underbelly of the 20102011 academic year has to be our failure to make AYPneither did Shelby County Schools or the State of Tennessee. Despite our AYP target, the MCS TVAAS data tells us a different story. We had dramatic improvements in mathematics in grades 38; dramatic gains in reading in grades 68; significant improvements in science in grades 6 and 7; improvements in Algebra I have stimulated requests to share approach; and 11th grade writing scores continue to impress. There is momentum in MCS that requires each of us to do our best to sustain it. This retrospective is a testimony to what we can do when we do not submit to the distractions of poverty, politics, and assaults on public education. We can do what we have done and more.

Irving Hamer, Jr. Deputy Superintendent of Academic Operations, Technology, and Innovation

Authors Note The present work is the second installment of a series of case studies documenting the implementation of Memphis City Schools teacher effectiveness reform. The first installment was a comprehensive account of the conditions of Memphis City Schools that warranted district-wide reform, particularly in the area of teacher effectiveness. The Year One (20092010) case study also reported early planning that laid the foundation for strategies that have the potential to transform the entire school district. The Year Two (20102011) case study chronicles implementation of teacher effectiveness reform as dictated by the four strategies that mandate that we: 1) create a common, agreed-upon definition of effective teaching; 2) make smarter decisions about who teaches our students; 3) better support, utilize, and compensate teachers; and 4) improve school culture and climate to foster effective teaching. This is an account of the preparation to move from planning to execution, and therefore, is designed to expand the context of reform in a way that provides a record of implementation with emphasis on our early attempts to pilot initiatives and lay the foundation for full implementation of strategies in years to come. More specifically, this iteration of our story of reform reflects concrete efforts to move from planning to implementation and continue to develop strategic and tactical plans to bring our work to scale districtwide. Unlike the previous case study, this document signals the increasing prominence of strategies related to policy, community advocacy, and communications that bolster our reform efforts beyond the four core strategies mentioned above. I remain committed to function in a participant-observer role to provide an insider or outsider point of view where appropriate. Similar to the first iteration of documented implementation, you will find that I continue to write in both first and third person, depending on my perspective of a particular aspect of the work. I intend to depict the realities of reform that equally resonate with those who are hearing about our work for the first time as with those who are closest to implementation. Now, more than ever, we are looking to our thought partners and stakeholders to impart a critical appraisal of our work as we embark on an educational landscape that is unknown to many and fairly new to others. We invite you to carefully read through, reflect on, and react to our course of action to date.

Kristin M. Walker, PhD TEI Archivist

ii

Table of Contents Strategy No. 1: Define and Measure Effective Teaching ............................................................................ 1 Context ...................................................................................................................................................... 1 Implementation and Findings ................................................................................................................... 3 Execute on the objectives of the Gates Research Plan (MET project).................................................. 3 Develop and implement each component of the TEM.......................................................................... 5 Conduct intensive training of MCS teachers and principals to improve awareness of value-added metrics ................................................................................................................................................. 13 Conclusion .............................................................................................................................................. 14 Ongoing Issues and Next Steps............................................................................................................... 15 Exhibit #1: Exhibit #2: Exhibit #3: Tennessee Teacher Evaluation Model: 2011-2012 School Year ........................................ 18 Teacher Effectiveness Measure (TEM) .............................................................................. 19 Tripod Survey Dimensions and Sample Questions ............................................................. 20

Strategy No. 2: Make Better Decisions about Who Teaches ...................................................................... 21 Context .................................................................................................................................................... 21 Implementation and Findings ................................................................................................................. 23 Improve recruitment and hiring of high- potential teachers through partnership with TNTP ......... 23 Better coordinate and leverage outside partner-ships that recruit and place high-potential teachers in MCS ................................................................................................................................................ 27 Raise the bar and improve the process for granting tenure ................................................................. 27 Increase the retention of effective teachers, particularly in early in their careers ............................... 28 Increase the turnover of our most ineffective teachers ....................................................................... 29 Conclusion .............................................................................................................................................. 32 Ongoing Issues and Next Steps............................................................................................................... 32 Exhibit #4: Exhibit #5: Smarter Decisions About Who Teaches: Staffing Progress to Date ................................... 34 Smarter Decisions about Who Teaches: Evaluation Progress ............................................ 35 iii

Exhibit #6:

Teacher Absences during the 2009-2010 School Year ....................................................... 36

Strategy No. 3: Better Support, Utilize, and Compensate Teachers ........................................................... 37 Context .................................................................................................................................................... 37 Implementation and Findings ................................................................................................................. 38 Improve the teacher evaluation process .............................................................................................. 39 Connect professional support opportunities to individual need .......................................................... 41 Create new and differentiated career paths ......................................................................................... 47 Implement a new base compensation structure ................................................................................... 47 Strategically place the best teachers where they are needed most ...................................................... 49 Cluster high-potential teacher recruits in schools with the most high-need students ...................... 49 Build a service oriented culture in the district toward teachers .......................................................... 49 Conclusion .............................................................................................................................................. 50 Ongoing Issues and Next Steps............................................................................................................... 50 Exhibit #7: Exhibit #8: Spring 2011 Reflective Practice Pilots ................................................................................ 52 2011 Survey Results on Teacher Support ........................................................................... 53

Strategy No. 4: Improve the Surrounding Context to Foster Effective Teaching ....................................... 54 Context .................................................................................................................................................... 54 Implementation & Findings .................................................................................................................... 55 Principal Leadership Capacity ............................................................................................................ 55 Improve school culture and climate .................................................................................................... 56 Develop a new technology platform ................................................................................................... 58 Conclusion .............................................................................................................................................. 59 Ongoing Issues and Next Steps............................................................................................................... 59 Other Strategies Related to the Success of TEI .......................................................................................... 61 Context .................................................................................................................................................... 61 Implementation & Findings .................................................................................................................... 62 iv

Influence and track policies to support TEI ........................................................................................ 62 Develop communications strategy around TEI ................................................................................... 63 Build community advocacy around TEI ............................................................................................. 64 Conclusion .............................................................................................................................................. 65 Ongoing Issues & Next Steps ................................................................................................................. 65 Enablers of Implementation ........................................................................................................................ 66 Barriers to Implementation ......................................................................................................................... 68 References ................................................................................................................................................... 70 Resources .................................................................................................................................................... 71 Acknowledgments....................................................................................................................................... 72 Department of Teacher Talent and Effectiveness Directory ....................................................................... 73

Define and Measure Effective Teaching

Strategy No. 1: Define and Measure Effective Teaching

integration of the project into the development of the TEM.

Context
MCS participation in the MET national research project was jump started in March 2010. MCS successfully executed the first year of the MET project when we recruited 447 teachers in 61 schools to participate. MET participants completed 94% (N = 2,384) successful video captures of teaching in the span of only three months. In addition to the video captures, MET teachers completed a hardpressed schedule for data collection. Data included measures of students performance on State standardized supplemental assessments; teachers content knowledge; students perceptions of the classroom environment; and teachers perceptions of the working conditions and support. During the first year of implementation, MET researchers judged Memphis an exemplar project site by the MET researchers for implementation because of our ability to engage a large number of teachers and get so much done in a short amount of time. Likewise, the original MCS project management team3 was lauded for their capacity to effectively manage schools and participants and to ensure that project requirements were met. Members of the TEI crossfunctional management team agree that the successful implementation was due, in part, to the singular focus on the work and the ultimate
3

The foundational strategy of the Teacher Effectiveness Initiative (TEI) is to create a common, agreed-upon definition of effective teaching. At the center of this strategy is Memphis City Schools (MCS) work to understand what effective teaching is from both conceptual and empirical lenses: MCS participation in the Measures of Effective Teaching (MET)1 Project and the development of the Teacher Effectiveness Measure (TEM), a multidimensional measure of teacher quality and performance. The TEI Case Study 20102 provided only a cursory explanation of the MET project to set the stage for a broader discussion about the MCS approach to defining and measuring teacher effectiveness. The alignment of the research project with the Districts work to develop a fair, objective measure of teacher effectiveness has since become more apparent, and, therefore, priority. Accordingly, a more robust description of the MET project is provided here to illustrate the importance of the
1

The MET project is a two-year national research project, supported by the Bill & Melinda Gates Foundation, which is designed to identify measures of effective teaching.
2

Walker, K.M. (2010). Case Study 2010: The Teacher Effectiveness Initiative.

The MET project management team for the first year of the MET (20092010) project included Tequilla Banks, Dr. Rorie Harris, Donna James, and Monica Jordan.

Define and Measure Effective Teaching follow through and delivery on what the team set out to do. More importantly, teachers feedback indicates that they are generally excited about the opportunities that come with participating in the research project, particularly their opportunity to engage themselves and their students in the reflective experience. Given the success of the MET project and its link to a multidimensional measure of effective teaching, it most logically follows that we would realize similar progress in the development of the TEM. All of the Year One (20092010) targets for the development of the TEM were met. Data sets (e.g., Mathematica value-added data, Praxis/ NTE4 scores, Tripod and Teacher Working Conditions surveys) for all components of the measure were secured and aggregated for initial testing to determine relationships among these data, especially their correlations with student achievement. (Recall that the aforementioned data sets reflect what is accessible at this time and have been utilized an early approach for understanding the relationships that exist among dimensions of the measure.) We have relied on Mathematica Policy Research (MPR)5 valueadded data in the absence of the States valueadded data (i.e., TVAAS6) in a format that the District can use in analyses needed for the TEM. According to the Tennessee Department of Educations website7, TVAAS is a statistical analysis of achievement data that reveals academic growth over time for students and groups of students, such as those in a grade level or in a school. TVAAS is a tool that gives feedback to school leaders and teachers on student progress and assesses the influence of schooling on that progress. It allows districts to follow student achievement over time and provides schools with a longitudinal view of student performance. TVAAS provides valuable information for teams of teachers to inform instructional decisions. TVAAS is not an additional student test, but a useful tool to help districts make data-driven decisions. Praxis/NTE scores continue to be the only available measures of teacher content knowledge and pedagogy. The MET project is supposed to begin a pilot of teacher content knowledge assessments in the second year of implementation. However, it is unclear whether the level of analyses that come from the research project will be useful for the purposes of the TEM. The Teacher Evaluation Working Group identified three observation rubrics (i.e., TAP, D.C. Impact, and a revised Tennessee Framework for Evaluation8) to be field tested for further development of the TEM. The Tripod and TWC surveys were administered districtwide at the end of 20092010 to collect stakeholder perceptions from students and teachers.

National Teacher Exam

7

Mathematica Policy Research (MPR) is the source of value-added data for the Effective Practice Incentive Community (EPIC) program in MCS. Additional information about MPR and the EPIC program are available at www.nlns.org.
6

Additional information is available at http://tn.gov/education/assessment/test_results.sht ml.

8

TVAAS is an acronym for Tennessee Value-Added Assessment System.

The Teacher Evaluation Working Group revised the Tennessee Framework for Evaluation during their summer 2010 retreat to prepare for the observation rubric field test.

Define and Measure Effective Teaching Year Two (20102011) milestones reflect plans to continue participation in the MET project as well as to finalize development of a measure of teacher effectiveness for roll-out for August 2011. The marriage between these two initiatives is important for MCS because we intend to leverage the MET project findings to refine and inform the TEM. The chief initiative for this strategy is to: Develop specific tools and implementation strategies for each of the major components of the TEM (i.e., growth in student learning, observation of teacher practice, stakeholder perception, and teacher content knowledge and pedagogy). This year, approximately 250 (of 447) teachers in 59 (of 61) schools are continuing their participation in the MET project. The teacher attrition is likely the result of changes related to time commitments, class schedules, and staffing adjustments in the schools. Despite this attrition, implementation of the MET research continues in the same manner as the previous year in that video-captured observations of practice, stakeholder percep-tions, and teacher content knowledge remain key points of measurement. The randomization of students to teachers is an added dimension to the methodology for the current year. Students were randomly assigned to MET teachers to explore whether students teacher assignments influence their achievement. The MET researchers provide a clear rationale for the importance of the randomization component of implementation: The only way to control for all the ways in which students differ is through random as-assignment, so teachers participating in the MET project have signed up as groups of three or more colleagues working in the same school, same grade, and same subjects" (pg. 11)10. I attended a November 2010 MET convening in Washington, D.C, which was of particular interest because it was slated as the meeting where Year One preliminary results would be presented to other research sites11. Specifically, the
10

Implementation and Findings

Execute on the objectives of the Gates Research Plan (MET project) The MET project is in the second year of implementation and is still on course with efforts to identify and understand elements of effective teaching. Since the start of implementation, the MET team has undergone personnel changes that should be noted here. When Tequilla Banks was instituted as the Executive Director of the Department of Teacher Talent and Effectiveness (DTTE)9 in July 2010, Monica Jordan (former research analyst in the TEM office) assumed the position of the MET Coordinator. Two research analysts, Anasa Franklin and Carole Anderson, were hired to assist with implementation for the remainder of the program year.

Bill & Melinda Gates Foundation. (2010). Working with Teachers to Develop Fair and Reliable Measures of Effective Teaching. Available at www.gatesfoundation.org/highschools/Documents/ met-framing-paper.pdf.
11

The Department of Teacher Talent and Effectiveness was formerly known as the Office of Teacher Talent and Effectiveness. The designation changed after Superintendent Cashs reorganization of Memphis City Schools in September 2010.

The following school districts are participants in the MET national research project: CharlotteMecklenburg Schools, Dallas Independent School District, Denver Public Schools, Hillsborough County Public Schools, Memphis City Schools, and New York City Department of Education.

Define and Measure Effective Teaching goals were as follows: 1) to understand preliminary findings; 2) to discuss potential implications of preliminary findings for the districts; 3) to strategize about communication with stakeholders; and 4) to understand districts plans for using videos after the study. I joined our MET and TEM teams in an all-day session with site updates, methodology (i.e., video scoring and software), preliminary findings, and next steps for the MET project. It was apparent that the implementation and application of the MET project intersected at various points for each of the school districts. Each project team had a distinct focus on aspects of teaching and learning that reflected the most salient issues of teacher effectiveness in their districts. Whereas some teams focused on integrating the MET project into plans for providing professional development experiences for teachers and administrators, other districts anticipated using the learnings from the MET project to inform career development and decisions for teachers. (As stated earlier, MCS intends to use what is learned from the MET project to inform our development of a new measure of teacher effectiveness). Our team also used this opportunity to talk about where our participation in the MET project has particularly advanced the development of the measure in the area of video observations. I came away with a new perspective of our work as it stands and going forward. MCS, along with the other project sites, is at the vanguard of teacher effectiveness reform. MCS cannot only learn from its own implementation of the MET project, but from the work of other districts available for adaptation to a Memphis model. Preliminary findings from the first year of the MET project were released in December 2010. A brief summary of findings is presented here to provide context and some justification for 4 building a new measure of teacher effectiveness for MCS teachers. The MET researchers stated that MCS data were not included in the preliminary findings because scores from our State tests were not available for analyses; the release of these scores was delayed due to changes in State standards and standardized tests. However, there is still something to be taken from the findings presented in the report to enhance the early development of the TEM. Preliminary results from the MET project shed some light on specific indices of effective teaching. The current discussion about MET preliminary findings is not meant to be extensive, since a more detailed account can be found in the official research report12. However, this discussion is intended to present the findings that are related to components of the TEM. Accordingly, some early findings suggest that: Teachers who lead students to achievement gains in one year or in one class tend to do so in other years and other classes; and Student perceptions of a teachers weaknesses and strengths are consistent across the different groups of students they teach.
12

Bill & Melinda Gates Foundation. (2010). Learning about Teaching: Preliminary Findings from the Measures of Effective Teaching Project. Available at http://metproject.org/downloads/Preliminary_Findi ngs-Research_Paper.pdf. Memphis City Schools Department of Research, Evaluation, Assessment, and Student Information (REASI). (2010). Measures of Effective Teaching (MET) Preliminary Results. From 30K: A Weekly Briefing from REASI, 3 (14). Available at http://www.mcsk12.net/aboutmcs_30K.asp.

Define and Measure Effective Teaching These findings support our work to develop a multidimensional measure of teacher effectiveness that includes student growth measures (e.g., value-added data), stakeholder perceptions, observations of practice, and teacher content knowledge. We anticipate similar outcomes from the initial analyses of the four components of the TEM. Develop and implement each component of the TEM The appointment of Tequilla Banks as the Executive Director of DTTE also precipitated personnel changes in the office that is responsible for the development of the new evaluation system. Dr. Rorie Harris replaced Ms. Banks as the Coordinator of the Office of Teacher Effectiveness Measurement, and Dr. Tracy Brittmon and Dr. Tracey Wilson were hired as Research Analysts for the Office of Teacher Effectiveness Measurement. This team is commissioned to conduct analyses of the TEM components (i.e., growth in student learning, observation of practice, stakeholder perceptions, and teacher content knowledge and pedagogy) which might be used to assess teacher performance within a multidimensional model. It is important to note that the development of the TEM was accelerated when changes in the Tennessee State legislature required every teacher to be evaluated each year with the new evaluation system, effective July 31, 2011. For MCS, this means that each year approximately 7,000 teachers have to be evaluated. Per state law, every educator in Tennessee will be evaluated using the following framework: 50% of the evaluation is measured by student achievement (i.e., 35% TVAAS data and 15% other measures of student achievement13), and
13

observations of practice account for the remaining 50% of the evaluation (see Exhibit #1). The TEM and the other State-approved models for evaluation are based on five scoring levels of effectiveness. The Teacher Evaluation Working Group crafted a recommendation for the TEM model. The MCS Board of Education approved the recommendations, thereby giving the District permission to present the TEM model to the State Board of Education on April 15, 2011. MCS alternate model for evaluation comprises 50% student achievement (35% TVAAS data and 15% other measures of student achievement), 40% observations of practice (measured by the MCS Framework for Teaching and Learning), 5% teacher content knowledge (measure to be determined), and 5% stakeholder perceptions (Tripod survey). The TEM model is depicted in Exhibit #2. Concerning the weighting assigned to the TEM components, the percentage of student achievement is in the statute; however, the percentages for observations of practice, stakeholder perceptions, and teacher content knowledge are subject to change depending on efforts to refine the model. MCS Deputy Superintendent Irving Hamer14 contends, We think these are formative, which means we can continue to learn more, refine more, and develop the measure more. The 5% [weight] for teacher content knowledge and stakeholder perceptions is an entry point for understanding how these components should factor into the model.
(e.g., EOC, ELDA, MAAS, TCAP Writing), school-wide TVAAS, student ACT/SAT scores, student graduation rate, student post-secondary enrollment rate, and completion/success rate in advanced coursework (e.g., dual enrollment).
14

The State-approved options for the 15% student achievement include: discipline-specific assessment

Irving Hamer is the MCS Deputy Superintendent of Academic Operations, Technology, and Innovation (AOTI).

Define and Measure Effective Teaching The use of TVAAS data to develop the TEM has been in a state of uncertainty, because we have not had access to the teacher-level data file to conduct analyses. (Recall that MCS teacher effectiveness work precipitated major changes to the State Legislature such that TVAAS data could be used for evaluation purposes. Despite these legislative changes, we have had to rely on Mathematica data via the EPIC15 program for analyses in the absence of the TVAAS data.) Efforts to secure access to TVAAS data have been ongoing given the datas pivotal role in the reform agenda. Deputy Superintendent Hamer visited the executive management team of SAS to discuss the possibility of a partnership with the organization to further the work and portfolio of MCS with regard to the use of valueadded data. The objective was to set the stage for changing the nature of the relationship that the District has with SAS as well as to establish precedent for interactions that would be in the best interest of MCS going forward. He reported that the meeting was promising and signaled some progress, yet the District made no official arrangements to secure the value-added data. Months passed after Deputy Superintendent Hamers meeting at SAS. The deadline for completing the TEM was quickly approaching without our confirmed access to the TVAAS data file. During that time, Tennessee Governor Bill Haslam appointed a new Commissioner of Education, Kevin Huffman, with whom Superintendent Cash and Deputy Superintendent Hamer arranged sessions to present the Districts comprehensive reform agenda. The need for urgent attention to the access and use of the TVAAS data as the foundation of our reform was a focal point of the discussion. Commissioner Huffman
15

On June 14, 2011, the Tennessee State Board of Education approved three alternate evaluation models, including the Teacher Effectiveness Model (Memphis City Schools), Teacher Instructional Growth for Effectiveness and Results (TIGER Association of Independent and Municipal Schools), and Project COACH (Hamilton Country Schools). The State has committed that TNCRED (Nashville) will be the external evaluator for all State-approved models to facilitate the refinement of each evaluation model. This account of the progress for building the TEM summarizes the decisions made with respect to each component of the TEM as well as the implications for preparing stakeholders for district-wide implementation virtually one year ahead of the schedule outlined in the original proposal (August 2012). The complexity of developing the TEM necessitates in-depth conversations about the individual components and the implications for execution in the years to come. For this reason, separate accounts of the planning and the progress around each component are provided here. Growth in Student Learning The utility and benefit of value-added data in assessing student growth and teacher performance are foremost in the rationale for inclusion in the multidimensional measure of teacher effectiveness. Not only is value-added data useful for predictive and longitudinal profiles of student achievement, but it is also an accountability measure for teachers and principals who have the most direct impact on student achievement. For this reason, the use of TVAAS data is foundational to our human capital work. 6

Additional information about the Effective Practice Incentive Community (EPIC) program is available at www.nlns.org.

Define and Measure Effective Teaching expressed a clear interest in the reform work of MCS and made a commitment to secure the TVAAS data for school districts across the state. For MCS, everything rides on this commitment from the Tennessee Department of Education. We cannot build or implement our TEM without unfettered access to this database. Observations of Practice Classroom observations provide critical insight into the quality of instructional practice. Accordingly, the Tennessee Legislature stipulates that observations of practice will count for 50% of the teacher evaluation. Further, the law also states that probationary teachers should be observed a minimum of six times for a total minimum of 90 minutes, and tenured teachers should be observed a minimum of four times for a total minimum of 60 minutes. Whereas observations of practice account for 50% of the States model for evaluation, observations of practice account for 40% of our TEM model. Within the framework of the TEM, it was important to identify an observation rubric that could serve as a basis for the common, agreedupon definition of effective teaching. The consensus among District leaders, principals and teachers was that the rubric should serve as a guide to understanding what explicitly was taking place in our classrooms. Accordingly, the first order of business was to identify an instrument that could capture the presence (or absence) of classroom behaviors that reflect effective teaching. The observation rubric field test was designed to identify the most appropriate observation model for a robust, multidimensional evaluation system one that is able to detect specific teacher behaviors that impact student achievement. The Teacher Evaluation Working Group was charged with the task of reviewing rubrics to be 7 included in the field test. This working group spent months reviewing nationally-recognized frameworks for observation and evaluation and made the final recommendation to use three rubrics for the field test: the MCS Revised Framework for Evaluation16, TAP observation rubric, and the DC IMPACT (TEACH domain). As plans unfolded for the field test and DC IMPACT became an infamous tool for evaluation across the nation17, we considered field testing only the MCS Revised Framework for Evaluation and TAP rubrics. However, the State Department of Education, which was conducting its own field test with the TAP rubric, asked MCS to proceed with testing the DC IMPACT observation instrument since that rubric was not a part of the State field test and could inform efforts for the State and our own research. The objectives of the observation rubric field test were as follows: To examine which observation rubric domains are the most sensitive detecting effective teacher instructional behaviors and resulting student actions; To understand which technology would be most suitable for agile data transmission to supply the necessary personnel with information to provide timely and appropriate support; To discover the most efficient process by which comprehensive observations

16

The teacher evaluation working group revised the original MCS Framework for Evaluation during its summer 2010 retreat.
17

The DC IMPACT rubric made national headlines for its role in terminating large numbers of teachers from DC Public Schools.

Define and Measure Effective Teaching may occur when scaling up to districtwide implementation in 20112012; and To receive comprehensive feedback from observers, teachers, principals, and teacher support personnel regarding the observation process. School principals (N = 28), assistant principals (N = 19), content specialists (N = 8), and regional office staff (N = 4) volunteered to be observers for the field test. Additionally, retired principals (N = 6) were hired as contracted observers for the field test. Dr. Brittmon recruited participants through personal contact via email messages, announcements in the Monday Memo18 and at monthly principals meetings, and correspondence to Memphis Education Association (MEA) membership. The pilot is designed such that school administrators, con-tent specialists, and contracted observers are conducting in-person observations. Contracted observers and regional office staff are also rating video observations using the three rubrics. The observers attended multiple training sessions on the rubric protocol, feedback delivery, inter-rater reliability, and technology use. Regional office personnel, content specialists, and contracted observers were trained on all of rubrics; and school administrators were trained on one of three observation rubrics for the field test. The observers responded favorably to the training sessions on the three rubrics. Members of the Teacher Evaluation Working Group attended some of the training sessions and agreed that the training sessions brought to light the fact that it is not easy to make judgments about an educators classroom practice. Likewise, they thought it was important for other teachers to
18

have this level of experience with the rubrics. This group made some recommendations for ongoing exposure to and training for the observation rubrics during the field test and going forward once that rubric is chosen for the TEM. The primary purpose of the pilot was to test three observation rubrics, but it also served as an opportunity to test potential technology tools that could facilitate future classroom observations. Observers were issued Apple iPads to maximize data capture during in-person observations. RANDA Solutions19 developed an observation field test application that was loaded onto the observers iPads. The field test application made the teachers names and rubrics available for real-time data collection and maintained records of progress for the observer. Despite wireless internet challenges in some schools, use of the iPad for data collection allowed the observers to complete the rubrics anywhere and sync the information to another computer/laptop when available. Observation reports were generated once the information synched to the server. The use of this handheld technology was seemingly appropriate for capturing and archiving observation data as well as providing timely feedback. However, some observers mentioned that they still wanted the capability to script their observation notes since the iPad did not have scripting capabilities. The iPad was chosen as the handheld device to be used to facilitate classroom observations. Laptop and desktop computers can also be used to access the online evaluation system. RANDA Solutions was selected as the vendor to create the electronic interface for the TEM and had a June 30, 2011, deadline for completion.

The Monday Memo is a weekly correspondence that is produced by School Operations to keep principals informed of District operations.

19

RANDA Solutions http://randasolutions.com/

Define and Measure Effective Teaching The selection of teachers to be observed during the field test was driven by the Districts existing evaluation schedule. Only teachers who were not already scheduled to be evaluated this year were eligible to participate in the observation field test. Human Resources generated a list of the teachers who should be evaluated this year, and this list was cross-referenced with the list of MCS teachers who were scheduled for evaluation to determine who would be randomly selected for participation. Teachers (N = 508) were randomly selected to participate in the field test. It was important to make the teachers who were randomly selected aware of the purpose and goals of the field test as soon as possible. The TEM office issued two memoranda one to the school principals and one to the teachers selected to participate in the field test. Whereas the memo to the principals was intended to inform everyone in the building about the field test generally, the memo to the teachers was intended to inform teachers that they were randomly selected to participate and outlined the expectations of their involvement. The observation field test began in January and ended March 11, 2011, and the TEM Office began analyzing the data and feedback from participants. They convened a group of the teachers who were observed using the three rubrics. Participating teachers were also asked to complete a feedback survey. The observers also participated in a feedback session. A summary of the feedback from both groups is provided here. Across all observer ratings, raters showed the strongest agreement (i.e., inter-rater reliability) when using the IMPACT rubric. The ratings for principals and content specialists yielded the strongest agreement with the IMPACT rubric. Current and retired principals demonstrated the most agreement when using the MCS Revised rubric. This level of agreement is likely due to 9 their breadth of experience using the current tool for evaluation. About 74% of the observers preferred the use of one of the field test rubrics over the current MCS observation rubric. Teachers also provided valuable feedback regarding the rubrics used in the field test. Nearly 68% of teachers reported that they preferred one of the tested rubrics over the current rubric being used to conduct classroom observations. The Teacher Evaluation Working Group used feedback from field test participants to arrive at a recommendation for an observation rubric for the TEM. The working group chose the IMPACT rubric to be a part of the TEM. Teachers and principals noted that the IMPACT rubric was straightforward and less cumbersome than some of the other rubrics they have encountered throughout this process. Likewise, the IMPACT framework has differentiated rubrics for certain teacher groups (e.g., art, physical education, special education). Upon final approval of the TEM model, the developers of the IMPACT rubric will begin the development of an IMPACT-based rubric for MCS so that it is customized to the observational needs of the District. The next step for preparing for increased numbers of observations and more frequent feedback to teachers is to identify personnel in the schools and across the district who are eligible to conduct classroom observations. According to Tennessee law, observers must be certified through State-approved training. Past practice in MCS has been also to require that observers have an administrative license. Because those who meet these criteria vary across schools, the TEM office asked principals to think about and communicate who could assist with observations in their buildings. Stakeholder Perceptions

Define and Measure Effective Teaching While value-added as a measure of learning is something that we care a lot about, we also care about the development of healthy dispositions and the quality of life in the classroom." Dr. Ron Ferguson20 As evidenced by preliminary MET findings, student perceptions of teaching and learning experiences are good indicators of teacher effectiveness. The TRIPOD survey21 was administered district-wide at the end of the last school year to collect students perceptions of classroom experiences from students in grades three through twelve. One of the most important lessons learned from last years administration of the student perceptions survey is the need to shore up the communication with teachers, principals, and others about the TRIPOD survey mainly the format, purpose, and utility. Teachers who participated in the MET project were most familiar with the Tripod survey, but they represented a smaller subgroup whose understanding of the measure did not necessarily migrate to the larger teaching corps. This disconnect is the source of increased levels of anxiety about the inclusion of stakeholder perceptions in new evaluation models. There are still many questions about the validity of having students judge their teachers as well as how much weight this type of feedback should hold when making decisions about teacher performance. Much of the confusion lies
20

in the lack of knowledge around the types of items built into the Tripod survey. For example, many teachers believe that the survey asked students questions to gauge how much they liked their teachers. Further, teachers were concerned that the responses to these types of questions would have implications for whether or not they might keep their jobs. Dr. Harris contends, Correcting the breakdown [in communication] is a big priority this year. We have to do a better job of dispelling the myths that surround the Tripod survey. One such attempt to dispel myths about the Tripod survey occurred during a TEI Institute22 in May 2011. Dr. Brittmon held breakout sessions about the Tripod survey to present information about the student perceptions instrument and answer any clarifying questions from teachers. Dr. Brittmon asked the teachers if they thought students could provide accurate and thoughtful perceptions of what was happening in the classroom. Interestingly, the overwhelming majority of teachers agreed that students could identify what was happening (or not) in their classrooms with respect to teaching and learning. Yet, their understanding of the value in having this type of stakeholder feedback was initially outweighed by the concerns for having students evaluate them. Upon entering the session, teachers received a K-W-L worksheet so that they could organize what they know, what they wanted to know, and what they learned about the Tripod during the session. Teachers revealed many misconceptions when asked to state what they already knew about the Tripod survey. Generally, they were aware that the Tripod survey had been used in
22

Dr. Ron Ferguson (Harvard University) is the developer of the TRIPOD survey.
21

The TRIPOD survey was administered and analyzed in partnership with Cambridge Education and Dr. Ron Ferguson. Additional information about the Tripod assessments is available at http://www.tripodproject.org/index.php/services/se rvices_surveys/.

TEI Institutes are half-day sessions held to train teachers, assistant principals and others on different aspects of the reform work.

10

Define and Measure Effective Teaching the MET project and allowed children to assess their teacher. They also knew that it was administered last year. Teachers expressed great concern that they could be fired from their jobs based on the student ratings provided. Likewise, they believed that students, having knowledge of the high-stakes nature of their input, would intentionally provide low ratings for teachers, particularly if they were at odds with the teacher during the survey administration. Dr. Brittmon presented the constructs23 that make up the Tripod survey to demonstrate that the survey does not assess students opinions. Rather, it measures students ratings of teachers behavior in the classroom. She reiterated that the questions on the survey do not ask students whether they like or dislike a teacher. The Tripod dimensions and sample questions are depicted in Exhibit #3. Teachers were also concerned that the length of the surveys would influence students responses. They asserted that students would only get to number 20 or so before they started just marking anything. Admittedly, the length (i.e., nearly 80 items) of the Tripod survey is a challenge, and the TEM office is exploring potential modifications to the length and administration of the survey. At the close of these sessions, teachers acknowledged their clear anxiety about the perceived subjectivity of the measure but confirmed that they learned a lot about and have a better understanding of the Tripod survey. Teachers feedback is reflective of the ongoing measurement and process refinement needed to mitigate their concerns and improve the use of the measure going forward.
23

The TEM Office is also working on ways to incorporate parent and peer perceptions into the model. (Recall that the original proposal stated that student, peer, and parent perceptions would account for 15% of the evaluation.) The District has struggled with having adequate measures and response rates on surveys from these groups of stakeholders in the past. The TEM office recognizes the 5% weight recommended by the Teacher Evaluation Working Group as an opportunity to explore appropriate measures and methodology for gathering peer and parent perceptions for inclusion in the teacher evaluation model. For this reason, student perceptions will be the only type of stakeholder perceptions for the initial roll-out of the TEM. Although the Tripod was originally scheduled for this spring, the administration has been rescheduled for early fall 2011 to align better with the data collection for other components of the TEM. This delay also provides a window of opportunity to explore different versions of and methods for the Tripod, namely those that allow younger groups of students (e.g., kindergarten to second graders) whom the original Tripod survey does not accommodate. Teacher Content Knowledge Teacher content knowledge is the least confirmed TEM component at this time, because there are dimensions to measuring this component that do not lend themselves to practicality, feasibility, or consensus in implementation. The majority of stakeholders agree that the use of certification exams (e.g., Praxis, NTE) is static and not necessarily representative of the teachers ability to demonstrate knowledge in their content areas and move children toward student achievement. The only data we have available for early testing of the TEM components were Praxis and NTE data; however, scores from these tests were 11

The Tripod survey is built on the Seven Cs which refer to the teacher competencies or behaviors. The Seven Cs are: Caring, Controlling, Clarifying, Challenging, Captivating, Conferring, and Consolidating.

Define and Measure Effective Teaching included only during preliminary testing to assess potential relationships between teacher knowledge and student outcomes. Preliminary analyses from the TEM office indicated that Praxis scores are not correlated to student achievement. MET project researchers are piloting a content knowledge assessment in conjunction with ETS. To date, 82 MCS MET fourth and fifth grade English/Language Arts and Mathematics teachers have taken the content knowledge test. Ideally, data collected from these teachers content knowledge tests would inform our efforts to include this dimension in the measurement model; however, the survey administration through the MET project is intended for grouplevel analyses to inform professional development. Thus, these findings are not generated in the teacher-level format that is needed to incorporate in our multidimensional model. With these challenges and the deadlines for TEM implementation imminent, the question remains whether we can test content and pedagogical knowledge as evolving dimensions practice. When initial strategies for measuring teacher content knowledge did not emerge as expected, the TEM office looked to the teachers for input. In April 2011, the TEM office held a series of focus groups with teachers to collect feedback on the utility and measure of content knowledge for teacher evaluation. Forty teachers attended a series of focus groups to provide feedback on measures of content knowledge for the TEM. They represented core content courses as well as non-tested subjects across all grade levels (i.e., elementary, middle, and high school). Dr. Tracey Wilson, Research Analyst for the TEM office, invited teachers to participate in conversations about the teacher content know12 ledge component. Teachers received questions ahead of time to prepare for the focus group discussions. She opened the sessions with a review of how all four components (i.e., student growth data, observation of practice, teacher content knowledge, and stakeholder perceptions) are incorporated into the TEM. Dr. Wilson also presented the results of a survey on potential measures of content knowledge. The survey results indicated teachers preferences for using Praxis (15.9%), standardized tests (11.3%), content-specific observations (37.8%), portfolios (28.4%), or a combination of these options (6.5%) as measures of content knowledge and pedagogy. The impact of teacher content knowledge and pedagogy on student achievement, teacher content knowledge assessments, and other measures of content knowledge were the topics of discussion for the focus groups. Teachers agreed that teacher content knowledge and pedagogy were very important for student achievement. The consensus was that, you cannot teach what you dont know, and you can use better strategies if you know and feel comfortable with the content. Teachers also agreed that content knowledge and pedagogy should be measured separately, because they require a different set of skills. Teachers identified several problems with using Praxis (or NTE) scores to measure content knowledge. One of the most frequently mentioned problems was the fact that Praxis is a snapshot of teachers content knowledge. Teachers in every session argued that many people do not test well, and a test would be an inaccurate reflection of what they know. For example, there are teachers who might test well but might not implement good strategies in the classroom. Teachers recommended different formats for testing if it had to be an option. For example, open-ended questions may help assess

Define and Measure Effective Teaching pedagogical strategies, and perhaps task-oriented or multiple-choice questions could assess knowledge and skills. Completion of additional college courses in specific subject areas was also suggested as a measure of content knowledge, with the caveat that this type of assessment may be more appropriate for secondary school teachers. Portfolios were a popular choice; however, teachers said that portfolios could still be subjective and should perhaps be assessed using a rubric. Teachers across all of the focus group sessions preferred a combination of measures (e.g., portfolio, assessment, content-specific observation) for content knowledge and pedagogy. They supported the differentiation of measures in the way that instruction and testing are differentiated for students. To this end, the TEM office convened teacher working teams during the summer to identify and/or design measures for content knowledge and pedagogy. Because the menu of options for measuring content knowledge might likely be limited, the TEM office and teachers will also examine existing practices (e.g., National Board Certification) for demonstrating content knowledge. Conduct intensive training of MCS teachers and principals to improve awareness of valueadded metrics Beyond a series of archived WebEx training videos on student growth data that are available through the MCS Mediasite24, the District does not have the capacity to deliver the level of training needed to improve awareness and understanding of value-added metrics. There are several factors that limit the Districts capacity to expand a common sensibility of value-added
24

data, not the least of which is the ambiguity of the definition and utility of value-added metrics for various stakeholders (i.e., teachers, principals, and administrators). The potential uses of value-added data are far-reaching and vary depending on the need and interest of the stakeholder. For example, teachers can use their own as well as their students value-added scores to reflect on their practice and use students value-added scores to drive instruction in the classroom. School administrators can use the value-added scores of teachers and students to engage in strategic planning and team building for their schools. We acknowledge the need to train District personnel particularly teachers and students on the differences between AYP (adequate yearly progress), as defined by the No Child Left Behind benchmarks, and value-added data. For example, many teachers in non-tested subjects are anxious about the use of their schools value-added score for their 35% measure of student growth, arguing that their evaluation score would suffer because their school did not make AYP. It has been a challenge to address teachers beliefs that achievement, rather than growth, is only one component included in the four-dimensional TEM model. The misconception and misunderstanding of value-added data versus AYP is a function of the historical practice and propensity for AYP designations in discerning performance. The District is partnering with a national educational reform organization to build the knowledge base for value-added data across the school district. Battelle for Kids is a not-forprofit organization that has contracted with the State of Tennessee to provide to districts across the state a variety of services related to teacher

Mediasite (mediasite.mcsk12.net) is the MCS repository for online professional development videos and materials.

13

Define and Measure Effective Teaching effectiveness. According to their website25, Battelle for Kids maintains a strategic program of work with regard to human capital reform: We partner with state departments of education, school districts and education-focused organizations to advance these strategies with the shared goals of: improving teacher effectiveness and student progress; informing instructional practice in real time; recognizing and rewarding teaching excellence; and aligning goals and maximizing impact in schools. In Memphis, the organization made training around value-added data available for teachers, principals, and district administrators. Battelle for Kids is working with the Office of Professional Development and Staff In-Service as well as the Office of Research, Evaluation, Assessment, and Student Information (REASI) to deliver training on value-added metrics. We are building district-wide capacity through a system of online courses for teachers such that 12 (of 26) hours of online training designated as mandatory professional development for teachers to deepen the understanding and utility of value-added data for guiding instruction and decision-making. Most of the teachers who have completed this training have reported that the training was very useful in helping them understand value-added data better. Even teachers without value-added data have seen the training and reported that it was helpful to them as well. The training has helped to make clear the distinction between proficiency and student growth as evidenced by the growing number of teachers who recognize that an adequate yearly progress (AYP) is not synonymous with their capacity to provide children with one years growth in achievement for one years instruction.
25

Training manuals were prepared for and disseminated to principals, regional superintenddents, and other District personnel to orient them to the purpose and utility of value-added metrics for improving student achievement. In December 2010, all principals attended two halfday training sessions on formative instructional strategies and value-added data with Battelle for Kids; the follow up training was held in January 2011. To date, 58% of certified educators (e.g., principals, assistant principals, instructional facilitators, and teachers) have completed the Battelle for Kids training across the district. All educators completed the training by August 1, 2011. The utility and application of value-added data are in the eye of the beholder. Whereas the Professional Development and Staff In-service office is coordinating the training for valueadded data in for guiding instruction, Dr. Harris in the TEM office is leading the charge to increase awareness of value-added data for teachers and principals as it relates specifically to teacher evaluation and aspects of career management. Because growth in student achievement is a major component of the TEM (and the States evaluation for that matter), it is imperative that teachers and principals have a clear understanding of what it means to factor student achievement growth into an appraisal of teachers performance.

Conclusion
The work around identifying a common, agreedupon definition of effective teaching continues to evolve as we learn more about what effective teaching looks like and explore the implications for teachers, classrooms, and school buildings. The MET project represents burgeoning research in public education reform, and our continued participation in the project guides the ongoing development of a multidimensional measure of 14

More information is available on the Battelle for Kids website www.battelleforkids.org.

Define and Measure Effective Teaching teacher effectiveness. We look forward to learning more from the MET project and from its expanded and continued implementation with the MET sub-study. The Bill & Melinda Gates Foundation funded an extension of the MET project that will emphasize the role of professional development in increasing teacher effectiveness. The hallmark of the MET substudy is the use of video coaching as a professional development tool. The MET team has already begun to introduce the MET substudy (i.e., the MET extension). The MET substudy is described in more detail in Strategy No. 3: Better Support, Utilize, and Compensate Teachers, because it is a major component of our work supporting teachers through reflective practice. The nature of teacher evaluation has changed for good, particularly across the state of Tennessee. MCS represents one of the school districts seeking to pioneer innovative evaluation models to increase teacher effectiveness. State law mandates 50% of teacher evaluation is student growth and achievement, but for its share, MCS has exercised autonomy to create a multidimensional model. We have also made certain that the development of the TEM has been a teacher-driven process. Teachers made decisions on which rubric to include in the model as well as how much each component should be rated in the first version of the TEM. By statute, school districts must revisit and refine their evaluation models on an annual basis. Therefore, it is in our best interest to keep teachers involved in the evolution of this work. Likewise, it is important to stay abreast and maintain focus on incorporating the most appropriate research and data for the components of the measure. Colleen Oliver26 reminded us that it is important to continue forging on in our original intent to build the TEM. She was emphatic in her support for the development of the measure in saying, The TEM is everything. It is about teacher control [over their evaluation]. The power of this [tool] is that it is grounded in research. I think it is a game changer. I think this sets Memphis apart from everybody else. MCS submitted the application for approval for the TEM to the State Board of Education on May 2, 2011, and the implementation plan for the model on May 16, 2011. Any school district in the state can adopt the TEM model once designated a State-approved model for evaluation.

Ongoing Issues and Next Steps

There are still some issues with the TEM that require urgent attention pursuant to the deadlines for final State approval and roll-out of the new evaluation system for the coming school year. Among the most pressing issues is identifying student achievement (15%) and teacher content knowledge (5%) measures and conducting district-wide training for teachers and principals for the TEM. Though access to teacher-level TVAAS data is imminent, the delay thus far has placed us in a vulnerable position for next year. To explain, there remain some inconsistencies with a proposed TEM profile because of not having early access to the TVAAS data to use to explore different evaluation scenarios and prepare for roll-out in the fall semester. Only about 35% of MCS teachers are in Statetested subjects. Therefore, there is a larger body
26

Colleen Oliver was the MCS Program Officer for the Bill & Melinda Gates Foundation until May 2011.

15

Define and Measure Effective Teaching of teachers for whom there is no measure of student growth in achievement (i.e., TVAAS). For these teachers, Tennessee law mandates that their schools value-added data (35%) and the State-approved options for achievement (15%) account for the student achievement portion of their evaluation. Similarly, new teachers who do not have TVAAS data will also have to use the school-level value-added data for their evaluations. Teachers are vehemently opposed to the use of their schools value-added data as a measure of student growth and argue that there are better ways to determine their capacity to advance students each year. The TEM office is working to understand better what the Tennessee law means for unique teacher groups and circumstances (e.g., interim teachers, support teachers, and schools without TVAAS scores) where student growth data are not available. The timing of the evaluation has many teachers and principals anxious. More specifically, the anxiety is a result of the ambiguity around when certain data will be made available for the evaluation. Many are also concerned about the timing of the TVAAS data for the use of the TEM. We understand there will likely be a oneyear lag time for TVAAS data. The timing of the release of data to the District is what engenders some apprehension for the TEM office (which is responsible for conducting analyses and producing evaluation profiles for every teacher, every year) as well as for the teachers and principals who wait in anticipation for the results of the evaluation. Decisions about how to measure teacher content knowledge warrant urgent attention. As it stands, we do not have a clear sense about what measure (or proxy) is most appropriate for understanding teacher content knowledge. The main challenge is solidifying an approach and having something that is a reliable measure of teacher content knowledge in place by the time 16 the TEM rolls out in August 2011. The State and other district stakeholders are not strong proponents of building separate measures for this component. The planned summer sessions with teachers helped to identify measures and methods that are appropriate. Addressing concerns around the Tripod survey particularly the student sample and the timing of administration of the survey are also a priority as TEM development continues. There is still no way of collecting the perceptions of students in the early grades (i.e., kindergarten through second grade). How can students in these grades provide feedback about their experiences in classrooms in a way that is valid and reliable? The validity of measuring student perceptions across multiple classes should also be considered. In other words, are the ratings of classroom experiences stable for teachers across multiple class periods? To administer the Tripod during one class period in elementary grades typically means that the survey is given to students who are with the same teacher for a significant portion of the day. Validity is more likely to be an issue in secondary grades because teachers have different classes of students throughout the day. There are also some questions about the amount of time needed to analyze the survey scores when administered at the end of the school year. What is the turnaround time for data to be used in reports of teacher performance? Cambridge Education, the vendor for the Tripod survey, is working to help streamline the administration of the survey and think through ways to ensure timely analyses and use of the survey data. With the exception of teacher content knowledge, the TEM model is in place; however, there are gaps in the outline of the process that support the tool. There has been limited discussion about what happens once a TEM profile is generated for teachers. Principals, in particular, have con-

Define and Measure Effective Teaching tinued to ask, Then what? They are referring to our need to articulate better the larger framework and implications for teacher evaluations. We have to determine who, other than the principals and assistant principals, can conduct classroom observations. It is also not clear what happens to a teacher who receives a below or significantly below expectations rating. Also, in addition to building-level logistics for evaluating teachers, there are several questions about the role of compliance (e.g., Human Resources and Labor Relations) in the new evaluation framework. These and other questions will guide training efforts to provide targeted responses and guide-lines for the processes that surround the TEM implementation. One other heavy lift for us is preparing to train all principals, teachers, and select District personnel on the evaluation system the observation rubric in particular. Insight Education Group, the framework developers, completed the rubric on schedule and began training different stakeholder groups (i.e., roughly 700 people) shortly thereafter. We also took advantage of two district-wide summer training opportunities The Forum for Innovative Leadership and the Practitioners Summit to begin to train principals and teachers, respectively. Training for all stakeholder groups will be ongoing as the measure continues to be refined and new developments unfold at the State level.

17

Define and Measure Effective Teaching

Exhibit #1:

Tennessee Teacher Evaluation Model: 2011-2012 School Year

Tennessee Teacher Evaluation: 2011-2012 School Year

35% - Student growth data (TVAAS) 15% - Additional student achievement data

18

Define and Measure Effective Teaching

Exhibit #2:

Teacher Effectiveness Measure (TEM)

Teacher Effectiveness Measure (TEM)

When evaluating teacher effectiveness, there are multiple lines of evidence to consider:

Student growth
Student achievement Observations of practice Stakeholder perceptions Teacher knowledge

19

Define and Measure Effective Teaching

Exhibit #3:

Tripod Survey Dimensions and Sample Questions

Measuring Student Perceptions: The Tripod Survey

The 7 Cs Sample Items My teacher in this class makes me feel s/he really cares about me Our class stays busy and doesnt waste time My teacher explains difficult things clearly My teacher wants me to explain my answers why I think what I think My teacher makes learning enjoyable My teacher wants us to share our thoughts My teacher takes time to summarize what we learn each day

Caring about students Controlling behavior Clarifying lessons Challenging lessons Captivating students Conferring with students Consolidating knowledge

20

Make Smarter Decisions about Who Teaches

Strategy No. 2: Make Better Decisions about Who Teaches

Increase the turnover of the Districts most ineffective teachers.

Context
Last year (20092010) marked the beginning of a new era for MCS recruitment, hiring, and staffing. The outsourcing of all recruitment and staffing services to The New Teacher Project (TNTP; also known as STARS27) was a bold approach to building a strong teaching corps in MCS. A key victory from this change in recruitment and staffing was the progress of the early staffing initiative for the Striving Schools Zone (SSZ)28. The Zone was completely staffed by the end of May 2010. Likewise, there were 1,705 total vacancies identified district-wide, and 1,550 (90%) positions were filled by the start of school in August 2010. This approach, though it yielded some success, was met with significant challenges. Changes to the recruitment and staffing strategies produced mixed results for the District. Despite some progress, nearly 100 vacancies were not filled on the first day of school. It is important to note the factors that facilitated the early staffing initiative. The success of the early staffing initiative with the SSZ was a function of partnerships with Memphis Education Association (MEA) and
27

Building a strong and effective teaching corps requires the delicate interplay of talent identification, performance management, and datadriven decision-making. The underlying assumption is that there are solid processes and policies in place to support these components that facilitate recruitment, staffing, and retention. Intuitively, the policies and procedures should be in place prior to enacting changes to the way teachers enter and matriculate through their careers in the District, yet Memphis City Schools (MCS) is working to do both in parallel to build a strong teaching corps and to establish appropriate processes and protocols to facilitate reform within the framework of human capital management. The milestones for this strategy are as follows: Improve the recruitment and hiring of high-potential teachers; Better coordinate and leverage outside partnerships that recruit and place candidates in MCS; Raise the bar and improve the processes for granting tenure; Increase retention of effective teachers, particularly early in their careers; and 21

As of January 2010, the Strategic Teacher Staffing and Recruitment (STARS) Office is responsible for all recruitment and placement of teacher candidates in Memphis City Schools.
28

Striving Schools is the designation given to highpriority schools (N = 28) in the district. Eight of the schools in the SSZ are in jeopardy of State takeover and categorized as the Achievement School District (ASD).

Make Smarter Decisions about Who Teaches partner programs. MEA granted staffing concessions for interim teachers and simultaneous job postings for internal and external candidates. Without the help of MEA in these areas, processes would have been further delayed. For example, the dismissal of interim teachers (whom many principals wanted to retain) could result in the loss of these teachers because of drawn-out processes of reapplying, interviewing, and onboarding. In years past, the District has lost many interim teachers to neighboring school districts because of the temporary contract or less seniority. Similarly, internal candidates have priority staffing privileges through the internal transfer process so that they are privy to vacancies before any external candidates can be notified for interview and referral for placement. The volume of applicants in 2010 (N = 1,800) who submitted applications to become teachers in MCS was due largely in part to the widespread recruitment strategies that targeted external teaching candidates. Likewise, coordinated efforts to work with partner programs to staff candidates in hard-to-staff schools and subject areas also influenced the influx of applicants. For example, Teach for America (TFA) supplied 100 teachers; Memphis Teaching Fellows (MTF) supplied 64 teachers, and Memphis Teacher Residency (MTR) supplied 26 teachers to support the Districts staffing goals. Generally, principals were pleased with the already apparent changes to and outcomes of the new recruitment and staffing process, believing they had access to better quality candidates and better opportunities to find the best matches for their schools. These sentiments are not surprising, given STARS commitment to external recruitment and mutual consent placement. Principals in the SSZ appreciated the urgency toward high-priority schools, because they have typically had to settle for candidates who may not be prepared or well22 suited for teaching in an urban environ-ment. Principals provided feedback on issues needing improvement going forward. Although principals had favorable comments about access to quality candidates, they were less complimentary in their feedback about the information and communication gaps between STARS and Human Resources. They were frustrated with having to go back and forth between STARS and Human Resources to get staffing updates and to have their questions answered. Further, disparate levels of information and access to data systems between the two offices left principals with no single source for staffing inquiries. This disconnect had direct implications for another limitation identified by principals a lack of responsiveness. The STARS team acknowledges that they were understaffed and could not adequately manage the volume of calls and emails coming into their offices from teaching candidates, principals, and other stakeholders. Ultimately, the Human Resources and STARS teams fell short of delivering good customer service to District stakeholders because of the quality of communication and collaboration between departments. Because STARS operations were a function of an information deficit, there were few opportunities for them to expedite many of the steps that were built into an already inefficient process. Many of the staffing challenges were brought about by a hiring timeline delayed by late identification of vacancies and cumbersome processes. (Late identification of vacancies is continuing to be a major problem insomuch that 78% of total vacancies were identified between May and September, and 60% [N = 717] of vacancies were identified across the district after July 1, 2010.) Moreover, 114 of these vacancies opened in August 2010. Those vacancies were typically classified as late teacher transfers,

Make Smarter Decisions about Who Teaches terminations (due to licensure), retirements, and resignations. There was very little traction as a result of targeted efforts to retain good teachers or to lay the groundwork for the dismissal of chronically low-performing teachers. A lack of clear workstream ownership, the commitment to traditional practice, and a marginal use of fragmented data sources contributed to the limited progress for this strategy. Difficulties staffing of the Office of Teacher Talent and Effectiveness (OTTE) 29 meant that there were no clear owners for the work of increasing the retention of effective teachers, of increasing the turnover of ineffective teachers, and of improving the tenure process. The tenure working group made a number of recommendations to the crossfunctional leadership team; however, no major changes resulted. It is clear that we needed to refocus our attention on increasing the turnover of our chronically low-performing teachers. Frankly, MCS did nothing to identify low-performing teachers and provide support to improve practice during Year 1, mainly because there was no objective means (i.e., a measure of teacher effectiveness) for determining what constitutes effective or ineffective teaching. processes. Much of what is described here with regard to recruitment and staffing has been the work of the Staffing Task Force. The work focuses on streamlining processes to maximize efficiency through an enhanced synergy between Human Resources and STARS. These two administrative entities are at a point where disjointed operations and communication pose threats to outcomes for schools, and most importantly, students. The launch of strategies to raise the bar for tenure is the product of the work of the Tenure Working Group and task force. Tenure has become a controversial topic of discussion because of a new State law that extends teachers probationary period to five years (versus three years). Moreover, teachers seeking tenure must receive above/significantly above expectations evaluation scores for the last two years of their probationary period. These changes, by virtue of increased standards for attaining tenure, have made the process more rigorous. Steps taken to raise the bar for granting tenure are inextricably linked to improving the evaluation process30, increasing the retention of effective teachers, and increasing the turnover of ineffective teachers. Accordingly, the overlap in the strategies and efforts are easily detected. Improve recruitment and hiring of highpotential teachers through partnership with TNTP Many of the strategies for making smarter decisions about who teaches include language about high potential teachers. In short, high potential refers to the teacher candidates who have come to the district through the partner programs (i.e., MTR, TFA, and MTF). External
30

Implementation and Findings

Major work has been done in the areas of streamlining operations and developing collaborative relationships among stakeholders (e.g., principals, staffing teams, partners, and District personnel) to improve recruitment and staffing
29

The Office of Teacher Talent and Effectiveness (OTTE) is now the Department of Teacher Talent and Effectiveness (DTTE). The change occurred after Superintendent Cashs departmental reorganization in September 2010.

Improving the evaluation process is a key initiative for the TEI Strategy No. 3: Better Support, Utilize, and Compensate Teachers.

23

Make Smarter Decisions about Who Teaches candidates who have been vetted by these programs are considered high potential based on the qualifications, selection, and training experiences that these candidates have prior to entry into the MCS teaching pool. This supposition is especially debatable, because it does not translate into the same claim for internal candidates for whom we are unable to pinpoint similar experiences. We will finalize the definition of high potential for internal candidates after the TEM has been rolled out district-wide. The primary Year 2 target for recruiting and staffing is to have 95% of vacancies filled by May 30, 2011, and to have the schools 100% staffed by the opening of schools on August 1, 2011. The Year 2 targets can be met only to the extent that there is an earlier staffing timeline and processes and transactions are consolidated. Given the targets set for the upcoming staffing season, early steps were taken to improve the collaboration between STARS and HR and to align work streams, tasks, and responsibilities among all departments involved in the recruitment and staffing processes. In September 2010, the staffing task force participated in a three-day staffing deep dive with Betsy Arons, an expert human resources consultant with the Bill & Melinda Gates Foundation. The purpose of the sessions was to identify areas for process improvements related to recruiting and hiring teacher candidates and to adjust milestones and staffing strategies for the 2011 staffing season. Ms. Arons opened the first session with a clear objective and disclaimer for the work that was before the team: Im going to push you pretty hard because the process is not a healthy one, and you all know it. There is a lot here than can be streamlined. You have a common ground; you want the best people, and you want them early. You are working hard, but you have a crazy process, and it is causing a lot of angst that 24 doesnt have to be there. Its bigger than working together in a bad process. The crazy process that Ms. Arons is referring to is the nearly 20-step process that begins with recruitment of potential candidates and ends with the on-boarding of new teachers. Several recommendations for establishing an earlier staffing timeline emerged from the staffing deep dives. The task force identified a more streamlined, collaborative working relationship between STARS and Human Resources: shared access to data, shared and equitable knowledge bases, and shared technological support for key processes for reaching the current years targets. Initially, it looked as if there were no changes to the staffing timeline because the proposed strategies reflected a staffing schedule that would begin in March 2011, as in years past. A memo to regional superintendents, budget center managers, budget services, STARS, and Human Resources detailing a timeline (November 30, 2010February 2, 2011) initiated the early staffing of budgeted positions. The memo outlined key events to guide and monitor progress toward early staffing. The timeline and related outcomes with implications for staffing protocols are described here. This summary of changes to the staffing timeline is not meant to be exhaustive, rather it is meant to highlight areas of the process that are new or have been consolidated to facilitate improvements in recruitment and staffing. In January 2011, principals received information to help them devise a staffing plan which was to include the following: enrollment projections, 20112012 master schedules, personnel information tied to the school, specialty positions (art, music, and special education), directives for budget cuts or additional position allocations, and a guidance document. Principals received the School-Based Budget and Staffing Guidance

Make Smarter Decisions about Who Teaches Handbook to help them identify their staffing needs at an earlier date. The document included a review of current staff and staffing allocations, including information related to the number of teachers to be surplussed (based on the staffing formula) as well as the number of grant-funded teachers whose funds end this year. This document also contained guidelines for surplussing teachers based on policy-driven criteria and procedures. For example, principals were encouraged to avoid the practice of using the surplus process to remove teachers who are not meeting performance expectations, because it is more appropriate to address performance through the teacher evaluation process. To address the challenge of late notification of vacancies, STARS launched the Declaration of Intent survey again this year to identify potential vacancies prior to the end of the school year. This year, nearly 70% of teachers responded to the survey to indicate whether they will retire from, transfer from, or stay in their current positions with the District. This year, 161 teachers have notified Human Resources about their decision to retire. Seventy teachers have been identified for termination for failure to meet licensure renewal requirements, and there are 40 vacancies due to resignations. In each case, the rate of notification does not align with the number of vacancies that have been identified as a function of retirement, termination, or resignation. The STARS team is working with principals to ensure that they identify and submit vacancies as soon as possible. Teachers had until June 30, 2011 to give notice of retirement or resignation. An electronic vacancy form was launched March 2011 for principals to use when reporting an opening at their schools. Members of the STARS team worked to create an electronic solution to address the need for faster turnaround times for vacancy identification and approval. In 25 the past, the principals completed a vacancy form and (manually) mailed it to Human Resources or STARS. The form was then sent to Budget/Finance so that the vacancy could be verified as a budgeted position. This vacancy form changed hands and moved from office to office several times, which typically resulted in misplaced paperwork and delayed turnaround times for principals looking to staff their buildings. The online vacancy form is basically an electronic replica of the paper-based form and can be accessed through the principal portal on www.teachmemphis.org. Once principals complete and submit the form through the website, the form is electronically routed to the Budget/Finance or Federal Programs department, depending on the funding source for the position. After personnel from these offices approve or reject the vacancy, a notification is sent to both the school principal and STARS confirming the vacancy and signaling STARS to begin referring candidates to principals for interviewing. Whereas turnaround times for the paper-based vacancy approval could take nearly three or four weeks, the expectation is that personnel at every step will honor a 24-hour turnaround time for action and follow up. The STARS team continues to develop mechanisms to monitor and set control limits at critical junctures to ensure that theses expectations are met throughout the process. This process is completely automated, including instances where principals need to submit additional documents (e.g., official notifications of retirement and resignation) from teachers in their buildings. There are still some areas that need adjustments as this is not intended to be the final solution for the technology and staffing needs of the District. Similar to the electronic vacancy form, the online transfer application was revamped to

Make Smarter Decisions about Who Teaches maximize efficiency and utility. The first transfer period began in early February 2011. Internal candidates accessed the voluntary transfer application through the teacher portal at www.teachmemphis.org. There were several challenges with the online application last year, including difficulties navigating the online process and submission of additional documentation beyond the online application. The STARS team has worked to improve the online interface for voluntary transfers based on teachers feedback. For example, the online transfer application was revised to be more user friendly, to include concise explanations of key terms, and to provide more frequent updates on the application status. Likewise, all of the information needed to process the voluntary transfer application is collected through this portal; teachers are not required to do anything other than complete the online application. There is still the matter of negotiations with the union on issues directly affecting recruitment and staffing. Voluntary transfer, seniority, and delineation of hard-to-staff schools were among the issues on the table for negotiation. MEA granted flexibility for staffing the SSZ. Similar flexibility was granted last year; however, feeder pattern and hard-to-staff schools were included in the population of schools (and subject areas) that would receive special considerations for staffing, such as simultaneous posting and interviewing for internal and external candidates and requiring principals to interview the top four (versus five) senior applicants during the transfer periods. The results of this collaboration signal the unions commitment to the District, and Deputy Superintendent Hamer is adamant in saying, MEA understands that if [they] dont do this, we cant get the [teacher effectiveness] work done. Hamer wrote similar comments in a recent article31 in the Tennessean stating: Nothing has happened in our schools without the awareness, participation, and support of the labor organization. For example, the Bill-&-Melinda-GatesFoundation-funded teacher effectiveness work underway here continues to be the object of deep partnership with the MEA. Apropos to these sentiments, the fruits of the early staffing planning and ongoing collaboration with MEA are quite impressive. During the first transfer period in February 2011, 70% of vacancies (N = 84) were filled. A comparison between April 2010 and April 2011 of the recruitment and staffing yields reflect the progress that has been made year-to-date. The number of candidates submitting applications to teach was 3,100 compared to 1,800 who submitted applications to teach in MCS last year. There was a 90% increase in the number of candidates ready for hire (N = 1,100) compared to the 21 candidates who were approved for the new teacher pool last year. Nearly 70% of the applicant pool received the highest quality scores based on the screening rubric and other qualifications. These data are summarized in Exhibit #4. As of June 2011, 90% of the SSZ and ASD schools were fully staffed, and the entire district was nearly 80% staffed. It is important to note that the STARS team used the same selection model that was used for recruitment last year. This level of progress suggests that there is a widespread interest in teaching in MCS despite the apparent financial and political challenges. We are confident that we are able to make smarter decisions about who teaches in our classrooms, because we have a larger, more qualified pool of candidates from
31

Hamer, I. (2011, April 28). Memphis schools make quick gains [Letter to the editor]. The Tennessean. Retrieved May 3, 2011, from www.tennessean.com.

26

Make Smarter Decisions about Who Teaches which to choose. We also attribute the increased number of applicants to the downfall in the economy, the improved internal processes, and the national attention on MCS reform agenda. The national attention on Memphis is salient, given that 45% of the new applicant pool is outof-state candidates. The boost in the number of applicants can also be attributed to the increased collaboration with our recruiting and staffing partner programs. perhaps new to the school district. The goal is to cluster teachers from partner programs in feeder patterns wherever possible to ensure that candidates from those programs are deployed as intended to fill high-priority District vacancies. For example, the partner programs are driven by missions to place their teachers in high-needs schools or schools that reside in the same feeder patterns. For example, MTR has 15 schools in which they want to place their resident teachers. We have committed to assist MTR and other partner programs in effectively clustering teacher recruits across the district. The capacity to cluster teachers in hard-to-staff schools and feeder patterns is also a function of the flexibility granted during MCS/MEA negotiations.

Better coordinate and leverage outside partner-ships that recruit and place highpotential teachers in MCS
Collaboration with the partner programs (i.e., TFA, MTR, and MTF) is critical to our recruitment and staffing work. We are committed to making partner programs priority to the extent possible. The partner programs have contributed to the successful recruiting and staffing efforts for the 20112012 school year, such that TFA recruited 150 candidates. MTR and MTF will each place approximately 2025 candidates. Collectively, the partner programs made agreements with the District to hire up to 226 teachers in high-priority schools and feeder patterns for the coming school year. Budget, position leveling, and surplus issues continue to challenge the extent to which the candidates who were recruited through these partner programs can be placed in schools across the District. Our strategy is to cluster teachers recruited through partner programs in schools that have high concentrations of high-need students. The goal is to have 70% of partner program teachers placed in schools with at least two other partner candidate teachers. Similarly, a broader goal is to have 70% of partner program teachers placed in a priority feeder pattern with at least eight other partner program teachers. Clustering serves to build in support networks for partner programs teachers who are new to teaching and 27

Raise the bar and improve the process for granting tenure
Prior to the passage of the aforementioned tenure law of April 2011, we were already on our way to increasing the rigor for granting tenure using the current evaluation process. There are 674 pre-tenure (i.e., one to three years of experience) teachers in MCS. Of the total number of pre-tenure teachers, 306 teachers were in their third year of teaching and possibly eligible for tenure, based on their years of service and licensure advancement. Deputy Superintendent Hamer charged every member of the TEI Cross-functional Management Team to make this group of teachers a priority for building a program of support and training to ensure rigorous evaluation and tenure decisions. Ms. Carla Holloway and Dr. Sherrish Holloman32 collaborated with the Professional
32

Carla Holloway is the Coordinator of Teacher Evaluation and Tenure, and Dr. Holloman is the Coordinator of Teacher Support, Retention, and Recognition for the Department of Teacher Talent and Effectiveness.

Make Smarter Decisions about Who Teaches Development and Staff In-Service (PDSI) team to leverage some of the New Teacher Induction sessions to reach out to these teachers and facilitate conversations about what steps to take to receive tenure. Data trends and feedback from teachers confirm that the evaluation process has not been implemented with fidelity, and it varies across school buildings. Ms. Holloway and Dr. Holloman worked with District personnel and administrators (e.g., Labor Relations, Regional Superintendents, Professional Development, Human Resources) to begin an aggressive campaign to inform and support teachers who are eligible for tenure. Support for pre-tenure teachers was an immediate priority as evaluation and tenure February deadlines were approaching quickly. The DTTE and PDSI held three workshops for pre-tenure teachers to receive professional development on various aspects of evaluation and tenure. The workshops covered domains of effectiveness to help teachers better understand the current framework for evaluation, including requirements, documentation, and outcomes. During these workshops, principals, teachers, and other school district personnel trained attendees on several aspects of teaching (e.g., portfolio, instructional strategies) that influence teacher evaluation. Ms. Holloway conducted several sessions that provided information and experience with the summative evaluation conference, documentation of teaching practice, and strategies for teaching domains. Approximately 200 teachers attended each workshop, and the general consensus was that the workshops were very helpful and informative. Teachers feedback also signaled the need to inform and support teachers as soon as they are hired to work. Specifically, their feedback exposed district-wide challenges related to the lack of information sharing and training available for pre-tenure teachers. For instance, teachers made 28 comments like, I learned several things that I should have been doing for three years. This information would have aided and benefitted my teaching not just the process [of getting tenure]. Another teacher responded, Excellent workshop! Although this is my third year teaching, this is the first time that I have a clear understanding of the evaluation process. The variability of understanding about evaluation among teachers is comparable to the knowledge and understanding of these processes among school principals.

Increase the retention of effective teachers, particularly in early in their careers

The aforementioned sessions held with pretenure teachers were representative of the kinds of strategies that we are exploring to increase the retention of effective teachers. For the first time, district-wide events were held to recognize and, ultimately, retain our teachers. In May 2011, the District joined 500 teachers in celebrating their attainment of tenure. The Tenure Working Group decided that the District needed to hold its first Celebration of Tenure event to offer recognition to teachers who achieve this milestone. The tenure celebration was the first retention strategy of its kind for acknowledging major accomplishments of teachers. The inaugural Prestige Awards event was held in March 2011 to recognize peer-selected teachers from each MCS school. The vision for the Prestige Awards is to develop selection criteria that are performance-based and aligned to the TEM while maintaining the peer selection. MCS and MEA co-sponsored an End of the Year Celebration for teachers. This event was intended to celebrate the close of a successful school year and show teachers that they are greatly appreciated. The tenure celebration, the year-end celebration, and the Prestige Awards are examples of early

Make Smarter Decisions about Who Teaches district-wide efforts to retain effective teachers. We understand that some retention strategies are unique to schools and have been successful at the school level. We are collecting information on best practices and exploring ways to bring these strategies to scale. Additionally, we intend to increase collaboration with partner organizations33 to share and implement strategies. detailed description appears in the case study for Strategy No. 3: Better Support, Utilize, and Compensate Teachers. The numbers reported here fluctuated for several months prior to final evaluation submissions. In fact, they were higher at earlier points in the term. When asked about the changes in the number of teachers slated to receive unsatisfactory evaluations, principals asserted that the teachers performance improved, so the status of their evaluation changed. According to Ms. Holloway, If a principal told me that the teacher improved in a matter of weeks, I asked them to outline what they did to help the teacher improve so quickly so that we could bottle it up and sell it to other principals. She reported that approximately 39 teachers improved after receiving support from their principals and District training opportunities. While there may have been some cases where intensive support from principals may have, in fact, improved teacher performance, it is also very possible that principals did not exit a teacher through the evaluation process. The declining number of unsatisfactory evaluations and non-reelections for employment is troublesome because it is symptomatic of the challenges to making difficult human capital decisions. It is understood that principals cannot make these kinds of decisions without the full support and backing of the District (e.g., senior administration and management, Labor Relations, Human Resources). How the District intends to operationalize this support for principals may still remain in question. For this reason, principals may still be unconvinced or discouraged about recommending teachers for nonreelection or submitting unsatisfactory evaluations. Initial conversations about the number of teachers who were being dismissed for unsatis29

Increase the turnover of our most ineffective teachers

There is no easy way to remove ineffective teachers from the classroom, so the steps to begin to make this happen required careful thought and attention. The first order of business was to ensure that rigorous tenure decisions were made using existing tools and processes (i.e., the current evaluation tools and processes). Fifty pre-tenure teachers were recommended for non-reelection (i.e., received unsatisfactory evaluations). Twenty-five tenured teachers received unsatisfactory evaluations. Of the recommendations for non-reelection of pre-tenure teachers (N = 50), 41 (82%) were actually dismissed. The number of non-reelection recommendations upheld represents a significant increase from last year when only 40% (N = 16) non-reelections were upheld. The number of tenured teachers to participate in tenure hearings is not confirmed at this time; these files are still under review. However, five (of the 25 reported) have been meeting with Labor Relations to determine if their cases will go to tenure hearings. These data are summarized in Exhibit #5. The processes that led to the increases in teacher dismissals were aligned to the strategy for improving the evaluation process, thus a
33

The TEI Partners represent business, education, and community organizations that support the MCS reform agenda. This group meets quarterly for updates on the work and strategic planning.

Make Smarter Decisions about Who Teaches factory evaluations were controversial, because progress toward raising the bar for tenure was left open for interpretation. There were many viewpoints considered before any conclusions were made. The numbers ignited great concern for many sitting in the conference room during a cross-functional team meeting. Someone raised the point that These numbers did not look to be different from what has been seen in years past, and if this is any indication of what will happen in February, then have we really raised the bar? It was absolutely the right question to ask, given the fact that only 38 teachers were non-reelected last year, and 50% (N=19) of these nonreelections were overturned by labor relations due to incorrect scoring or a lack of adequate documentation of performance. The question of our success at raising the bar for tenure based on these numbers, albeit interim, prompted much speculation. The responses ran the spectrum from the notion of having more good teachers than we initially thought to the question of whether we should even be trying to raise the bar for tenure at all. The first thought was that there were no significant changes in the numbers of teachers who earned (or were denied) tenure under a new, more rigorous regime. Deputy Superintendent Hamer argued, Our orientation, process, and infrastructure have changed; therefore, we should have demonstrable differences in the data. It was clear that he and others around the table were disheartened by the threat of reaching the status quo for another year. The back-andforth discussion and reactions to the number of teachers recommended for non-reelection in the meeting also revealed that there were no clearcut arguments to be made. Further, any potential confounds to the numbers of teachers up for non-reelection could be rationalized depending on various circumstances. For example, it is possible that scoring/rating being a function of 30 repeated use of the instrument contributed to the small numbers of teachers who received unsatisfactory evaluations. Lastly, there may more unsatisfactory evaluations submitted if principals felt supported in their making courageous decisions, and, in turn, those decisions were upheld. Ms. Holloway shared a different perspective. She said, The key is not how many nonreelections we have but whether or not they will be upheld. She continued to explain that the trend is to have most of the non-reelections overturned, so to have non-reelections upheld at a higher rate would be a clear indication of progress. At the core of Ms. Holloways statement was the fact that these kinds of process improvements have the potential for resulting change perhaps not radical change but change nonetheless. The process improvements to which she is referring pertain to the information and materials provided to principals in preparation for the 2011 evaluation cycle. At the very least, explanations offered here illustrate the importance of not taking a purely quantitative look at what is happening with respect to the evaluation practices, and, ultimately, tenure recommendations of principals. The varying claims and opinions of the progress notwithstanding, one thing is certain. What we were able to accomplish with regard to increases in the turnover of ineffective teachers was done when other school districts withheld making such decisions in anticipation of a new evaluation system. We also started to look at other factors that relate to teacher performance and student achievement. Historical data on teachers professional behavior (e.g., attendance, discipline referrals, and transfers) are the only sources of information available at this time. A data review revealed staggering trends in teacher behavior,

Make Smarter Decisions about Who Teaches particularly teacher attendance. Approximately 20% of teachers used more than 10 sick days during the 20092010 school year with 917 teachers absences falling in the range of 1120 days during the same year. These data were cleaned up only to include absences coded as illness by the payroll system and to exclude absences that reflect approved leave. Urgent and courageous action was taken by Deputy Superintendent Hamer and the DTTE staff to address attendance abuse. Over 800 MCS teachers were flagged for excessive absenteeism based on August 2009May 2010 payroll and Human Resources data (see Exhibit #6). These teachers received a letter from Deputy Superintendent Hamer that informed them of their attendance status and offered clarity around the available resources to improve these attendance trends. The letter was not an official reprimand. Rather, it was intended to raise the awareness of the pervasive attendance problem and to direct teachers to the appropriate District resources to address the issue. The letter asked teachers to call if they felt like they received the letter in error. For example, teachers were asked to contact Human Resources if their absences should have been documented as approved leave but had not been reported as such. Likewise, teachers were asked to contact Dr. Holloman if they needed help identifying support services for this issue. Principals also received correspondence with a listing of the teachers in their buildings who received the letters about the attendance trends of their teachers. The letter advised principals that the goal was not to provoke disciplinary action but to communicate the Districts objective to pay closer attention to these matters. Needless to say, as soon as the letter went out, a small firestorm ensued. The backlash from the distribution of the attendance letter made things a little uncomfortable, to say the least. Tequilla 31 Banks noted, If we didnt do this, we wouldnt have the courage to do any of the effectiveness work. Several District departments and MEA received calls from teachers who wanted to voice their concerns and report inaccuracies about the attendance data. DTTE staff members and personnel in other departments (e.g., Labor Relations and Benefits offices) were inundated with calls from teachers justifying their absences and reporting that they were, in fact, absent for legitimate (usually health and family-related) reasons. Many of these teachers mentioned that they had notified their principals of the leave and had documentation to verify the reason for the extended absences. The best explanation for these kinds of situations is related to the need to have access to clean, reliable data across the district. The initial run of the attendance data was done using the payroll and Human Resources systems, but these data systems do not cross-reference all of the systems in Human Resources that manage approved leave data. For instance, teachers who were on leave based on the Family and Medical Leave Act (FMLA) would not appear in the payroll database because the payroll system codes everything as a sick day, and there is no code for FMLA in that system. Further, application for leave can only be made when there are 15 consecutive absences or an amount deemed appropriate for intermittent medical leave. The number of teachers, who indicated that they were on an approved leave, though not officially, is a sign that there is no clear protocol for getting an approved medical leave. Further, if the protocol is clear, it has not been clearly communicated to teachers. To the extent that the proper documentation was submitted, any mistakes in attendance records were reconciled. Follow-up letters of apology were sent to 179 teachers who received the letter in error. The

Make Smarter Decisions about Who Teaches follow-up letter also included information that clarified the process for getting an approved leave. Comparative analyses of teachers attendance from August 2010May 2011, show that 388 teachers have been absent more than 11 days during this time span; 212 teachers were classified as repeat attendance offenders. We have not yet determined the consequences for the repeat offenders, but every effort is going to be made to monitor the data for every district employee, including teachers. attention to the factors that could lead to marginal performance and low student achievement. The work to date will be a solid point of reference and experience going forward.

Ongoing Issues and Next Steps

With budget cuts and lay-offs looming, the growing surplus pool of teachers is becoming a real issue. The greatest concern is the discrepancy between the number of surplus teachers and vacancies which is disproportionately affecting some subject areas (e.g., English, History, and some middle-school courses). In other words, we have more displaced teachers than vacancies. This has major implications for our capacity to recruit and hire external candidates and candidates from our partner programs in schools other than the schools and feeder patterns identified as highpriority and hard-to-staff. STARS plans to continue to collect data through exit surveys and/or interviews from teachers who decide to resign or retire. The coordination of partner programs and partner organizations is an evolving endeavor. As the needs of the District change for recruitment and staffing, the collaboration with these groups becomes an important resource. We will continue to staff schools with partner program candidates and to expand resources made available through these programs to teachers across the District, not just for teachers who are placed by the partner programs. The turnover of ineffective teachers is a complicated topic that highlights the need to address a litany of human capital and operational issues. While the District purports to establish a level of expectation for professionalism, the enforcement and consequences of these expectations are ambiguous. The District has historically maintained a progressive discipline plan for all employees, but it has never associated an 32

Conclusion
Whereas the strategy to make smarter decisions about who teaches our students fell short of accomplishment last year, with the exception of the recruitment and staffing work, there were many victories for a number of this years milestones. We made additional progress in recruitment and staffing by initiating an earlier hiring timeline and making interdepartmental processes more efficient. In the absence of our new evaluation framework (i.e., the TEM), we were able to increase the number of teachers (pre-tenure and tenured) who received unsatisfactory evaluations, thereby removing ineffective teachers from the classroom. Despite the apparent progress made for this strategy, there is still so much to be done with making better decisions about who teaches. Indeed, there are still positions that have been vacant since the beginning of the school year, and staffing remains vulnerable given changes in enrollment. The threat of State takeover and the budget crisis are also major barriers to progress. While outcomes of the recent changes to the recruitment and staffing processes are yet to be determined, the outlook is promising. Strategies used to increase the turnover of ineffective teachers are iterative at this point, yet they are a good starting point from which to begin paying

Make Smarter Decisions about Who Teaches expected attendance rate (i.e., percentage rate) to this plan and to do so is problematic. Moreover, sick days are accrued in accordance with the state-wide retirement system. The number of sick days allotted violates the Districts practice of having teachers be absent only 5% of the days during the school year. Most teachers view sick days as time earned and, therefore, believe that they cannot be punished for leave that is given to them by the State regardless of a District expectation or mandate. We are working to adapt tenure processes to align with the TEM evaluation tool and the new State tenure legislation. More specifically, we are considering establishing a Peer Assistance and Review (PAR) program for MCS. The Tenure Working Group and Tenure Task Force are beginning to identify appropriate elements of a PAR program to be implemented during the 20112012 school year. The disconnect between policy and practice on many of the strategies discussed here is costly and demands revisions to policies that address professional standards. In the not-too-distant future, every effort must be made to align policies with practice, align Human Resources data with policy, and develop training and communications for school- and district-level personnel on the policies and their implications. The Division of Policy and Legislation is working through the policies now, trying to make sure that teachers are held to accurate and appropriate standards.

33

Make Smarter Decisions about Who Teaches

Exhibit #4:

Smarter Decisions About Who Teaches: Staffing Progress to Date

Smarter Decisions About Who Teaches

Staffing Progress to Date:

April 2010
~ 3,000 candidates started an application 1,800 candidates submitted an application 85 candidates met the requirements of the Teach Memphis selection model 21 candidates submitted documentation and been approved for new teacher pool

April 2011
5,400 candidates started an application 3,100+ candidates submitted an application 1,900+ candidates met the requirements of the Teach Memphis selection model
90%+ increase in candidates ready for hire at this date 55% of the targeted total pool built 70%+ of the pool has quality score of 1 or 2 62% of pool has previous teaching experience

1,100+ candidates submitted documentation and been approved for new teacher pool

34

Make Smarter Decisions about Who Teaches

Exhibit #5:

Smarter Decisions about Who Teaches: Evaluation Progress

Smarter Decisions About Who Teaches

Evaluation Progress to Date:

SY 2009-10 Number of Pre-Tenure Teachers Evaluated Number of Pre-Tenure Teachers with Unsatisfactory Evaluations 1,361 40 (3% of all evaluations) 16 (40% of unsatisfactory evaluations) SY 2010-11 1,862 50 (2.7% of all evaluations) 41 (82% of unsatisfactory evaluations)

Number of Pre-Tenure Teachers Actually Dismissed (including upheld evaluations, resignations and dismissals)
Number of Tenured Teachers with Unsatisfactory Evaluations Number of Tenured Teachers Actually Dismissed

17 5 (29% of all unsatisfactory evaluations)

25 -(File Reviews still in process)

35

Make Smarter Decisions about Who Teaches

Exhibit #6:

Teacher Absences during the 2009-2010 School Year

4,000

Number of Teachers by Number of Absences, 2009-2010

3,304

3,000

2,304
Number of Teachers

2,000

1,000 571 148 0 0 195 1-10 11-20 21-30 31-50

738 teachers
146 51-100

16 101-150

5 151-200

Number of Absences

36

Better Support, Utilize, and Compensate Teachers

Strategy No. 3: Better Support, Utilize, and Compensate Teachers

Cluster high-potential teacher recruits in schools with the most high-need students; and Build a service-oriented culture in the District toward teachers.

Context
Very little progress resulted from the strategy to support, utilize, and compensate teachers during Year 1 of the TEI. A number of factors influenced the marginal success of this strategy, particularly the staffing of the Department of Teacher Talent and Effectiveness (DTTE)34. The absence of initiative owners was especially felt where team members were needed to pioneer work that had not taken shape in prior years, namely improvement of the evaluation process, development of professional support programs, and a new base-compensation schedule. The existing evaluation process and tool (i.e., Tennessee Framework for Evaluation) are completely observation-based and have not provided a clear and accurate reflection of teachers performance across the district. Historically, nearly 98% of MCS teachers receive a satisfactory evaluation. These performance ratings are inconsistent with the performance levels of many of our schools. Though a multidimensional measure of teacher effectiveness is imminent, there is an urgent need to improve the use and implementation of the existing evaluation process in the meantime.
34

This work centers on teachers, and we cannot continue to cavalierly fail to support, recognize, and retain our teachers. Irving Hamer Teacher support has become a major program of work with regard to teacher effectiveness. The consensus is that support strategies must undergird the teacher effectiveness reform. Support for teachers must be comprehensive, differentiated, and ongoing to appropriately inform any performance and career management decisions for teachers. A new milestone was added to this strategy for Year 2 as a result of the growing need to address several gaps in service and support to MCS teachers. The specific Year 2 milestones for this strategy are to: Improve the teacher evaluation process; Connect professional support to individual need; Create new and differentiated career paths; Establish a new base compensation structure; Strategically place the best teachers where they are needed most; 37

The DTTE was formerly named the Office of Teacher Talent and Effectiveness. The name change occurred as a part of Superintendent Cashs reorganization of Memphis City Schools in September 2010.

Better Support, Utilize, and Compensate Teachers Memphis City Schools repertoire for teacher support includes new teacher induction, a mentor program, and a menu of online and in-person professional development courses. Admittedly, the District falls short of providing teachers with professional support that is linked specifically to their individual needs. For decades, the MCS compensation schedule has been based on degree attainment and years of experience (i.e., steps) and designed such that a teacher has to work in the District approximately 18 years (i.e., 18 steps) to reach his or her maximum earning potential at approximately $65,000. Nearly 4,800 (of total 7,000) teachers have worked in the District long enough to reach this point on the compensation schedule. The existing compensation system lends itself to much scrutiny. First, the system is not attractive to teacher candidates in todays market. The District is not able to compete for top candidates because it cannot offer competitive wages and benefits within the current framework for compensation. Second, it implies that teachers are homogenous and operate at the same or similar levels of teacher effectiveness. Third, it asserts the underlying assumption that the teaching profession is static rather than a dynamic path of professional growth and experience. Very few MCS personnel have the knowledge and command for revamping the compensation work. Therefore, we have limited capacity to do the research and planning for building a new compensation schedule for 16,000 employees and 7,000 retirees. The opportunity to build an entirely new, performancebased compensation system is accompanied by an equally singular opportunity to differentiate the career paths of MCS teachers. Similar to the existing compensation schedule, teacher roles and career development are driven by years of service. Financial incentives that 38 teachers receive work like commissions wherein teachers are paid a pre-determined amount of money for additional responsibilities or involvement in various District-run programs or activities. The plan is to align the new basecompensation schedule with the differentiated career paths for teachers and have both of them work as performance-based systems.

Implementation and Findings

In July 2010, Tequilla Banks was appointed to be the Executive Director of the Department of Teacher Talent and Effectiveness (DTTE) after the resignation of Dr. David Hill35. Ms. Banks set her early sights on staffing the core teacher effectiveness staff. Within three months, Ms. Banks hired Lachell Boyd (Teacher Liaison), Carla Holloway (Coordinator of Teacher Evaluation and Tenure), Dr. Sherrish Holloman (Coordinator of Retention and Recognition), Jessica Lotz (Special Projects Coordinator), Marqui Fifer (Special Projects Coordinator for Evaluation and Tenure), and Jennifer Chandler (Special Projects Coordinator for Reflective Practice) to engineer the implementation of several key strategies of the teacher effectiveness work. The DTTE is supported by two executive assistants (Lee Brother and Miesha Turner), and Donna James is the research assistant for the team. Staffing of Year 1 positions is practically complete, with the recent hire of Mike Neal as the Coordinator of Career Management. As a result of strides made with staffing the DTTE, Ms. Banks and her team have jump-started much of the work that lay dormant during the last year.

35

Dr. David Hill was the Executive Director for the TEI 20092010.

Better Support, Utilize, and Compensate Teachers

Improve the teacher evaluation process

Because the new evaluation system (i.e., TEM) does not take effect until 20112012, emphasis had to be placed on improving practice and outcomes related to the current use of the Tennessee Framework of Evaluation. Ms. Holloway, a former elementary school principal, spearheaded the work to prepare principals to make better use of the evaluation tool and process. Disparate knowledge about the evaluation process and use of the tool among teachers and principals contributed most to the status of evaluations in MCS. Ms. Holloway led the charge to help principals conduct teacher evaluations with an increased level of fidelity. In the early fall, all principals were given a Principal Toolkit for Evaluation. The toolkit outlined specific deadlines, documentation, and other requirements for completing a teacher evaluation. The purpose of the principals toolkit was to provide principals with a step-by-step manual on how to conduct evaluations. It covered general topics (e.g., evaluation forms and deadlines), sample tools for classroom observations, and best practices for managing the evaluation process. The toolkit was disseminated to principals during the fall semester in time to begin their classroom observations. In addition to the toolkit, principals attended a series of training sessions on each of the toolkit components, including training for understanding the domains of effectiveness, completing classroom observations, conducting the summative meeting, and scoring the evaluation paperwork. Most of the training took place during the monthly principals meetings. Ms. Holloway also recorded Mediasite36 videos
36

entitled Conducting More Rigorous Teacher Evaluations and Navigating the Evaluation Process37 for principals and teachers, respecttively. Because many principals struggled to locate teachers past evaluation records and documentation, Ms. Holloway also facilitated the transfer of prior years evaluation files for teachers who have transferred from one school to another. Unlike years past, principals would have help in retrieving teachers previous evaluations to complete their files. Further, they would be able to use the files to understand better teachers past performance trends and the plans of improvement that were associated with those evaluations. This year, Ms. Holloway coordinated changes in the way evaluations were submitted and reviewed. She began working with principals in late October and early November 2010 to identify teachers who were likely to receive an unsatisfactory evaluation. The purpose of her request was two-fold. First, she wanted to help principals identify teachers who needed targeted support as a function of the results from their classroom observations. Principals submitted the names of teachers who could potentially receive an unsatisfactory evaluation to Ms. Holloway. She then worked with principals to develop plans of improvement for teachers who needed additional assistance and support in their practice. Second, she wanted to help principals to get a head-start on gathering and maintaining adequate documentation. Ms. Holloway and regional staff reviewed files and documentation
37

Mediasite (mediasite.mcsk12.net) is the Districts repository for online professional development videos and materials.

Conducting More Rigorous Teacher Evaluations and Navigating the Evaluation Process is available at http://mediasite.mcsk12.net/mediasite5/Catalog/pa ges/catalog.aspx?catalogId=f06e4d93-f18e-49acbbe8-1192588b28b5.

39

Better Support, Utilize, and Compensate Teachers before submitting completed evaluations to Human Resources to minimize technical errors that typically lead to overturned unsatisfactory evaluations. Principals reported that the additional training and support that they received during this evaluation cycle were very helpful. Principals reported that they received most of the information from the monthly principals meeting, but one-on-one conferences and Mediasite presentations were helpful as well. If all goes according to plan for the next school year, there will be no need to manually review and submit evaluation documents. An electronic evaluation system would improve the evaluation process by facilitating the process, storing data, and streamlining documentation. The pursuit of an electronic evaluation system to streamline and expedite the evaluation process began at the start of the school year. The purpose of an electronic evaluation system is to reduce the amount of paperwork and time involved in evaluating teachers, particularly in preparation for the State mandate to evaluate every teacher and principal in the district beginning 20112012. The Online Principal and Teacher Evaluation System (OPTES) is the electronic system designed to facilitate all educators (e.g., principals, assistant principals, and teachers) evaluations for MCS, namely the observation component of the TEM. RANDA Solutions, the company that assisted with the observation rubric field test, was selected as the vendor to build the OPTES system. The system will house an IMPACT-based rubric38, maintain a scheduling and management protocol, and archive relevant data to facilitate classroom observations. OPTES has website and iPad (or
38

Android) applications for ease of use. The OPTES system and the rubric were to be completed on a target date of June 30, 2011. Training on the evaluation framework (including the rubric) and the electronic system was set for July 2011. Principals were introduced to OPTES during their monthly training sessions as well as during the summer professional development experience for principals (i.e., Forum for Innovative Leadership). They were optimistic about its utility in managing observations. However, they had several questions about which documents are necessary for completing the evaluation in addition to the observation rubric. For example, the existing framework for evaluation required appraisal records, professional growth plans, educator information records, and summative documents. Ms. Holloway plans to survey teachers and principals to determine what documentation is appropriate for the new evaluation framework. Once determined, the documents will be included in the electronic system. There have been some challenges in the development of OPTES. The biggest challenge has been the timely completion of the development and testing phases. The original plan was to have OPTES ready for district-wide use, at least for pilot, during the spring 2011 evaluation cycle; however, no such pilot occurred due to changes in the requirements for the system and delays in finding a vendor to build the system. The challenges revealed that the deficiencies in the Districts technology infrastructure have major implications for our attempts to improve processes and practices. While the primary focus at this time for OPTES is classroom observation, the application could evolve to include value-added data and other teacher effectiveness data. The MCS OPTES team (composed of Carla Holloway and Isabella 40

The approved observation rubric for MCS will be a customized version of the IMPACT rubric that was selected at the close of the observation rubric field test.

Better Support, Utilize, and Compensate Teachers Wilson39) is thinking through ways to modify the online evaluation system in preparation for implementation of the TEM. teachers through face-to-face and virtual learning experiences. The PDSI team has coordinated the move toward more virtual learning experiences for teachers and other district personnel. In addition to expanding the online repertoire for professional development courses, the manner in which teachers participate in first-of-the-year professional development has drastically changed. For two years now, MCS has also worked toward delivering individualized professional support to teachers in the form of a district-wide professional development experience for teachers prior to the start of the school year, namely the Practitioners Summit. For the second annual event, nearly 7,000 teachers convened at Bellevue Baptist Church during the first week of August to attend sessions on various topics and content areas. They were able to select sessions that were relevant to their professional responsibility for the coming school year. The PDSI office has also worked to align current professional development offerings to the existing Tennessee Framework for Evaluation. More specifically, PDSI team created and disseminated a professional development resource guide as a tool that principals can use to help their teachers find the most appropriate professional development opportunities. In like manner, the PDSI team is working to create an updated resource guide that aligns professional development offerings to the MCS Framework for Teaching and Learning, with particular focus on the domains and indicators of the new rubric. Plans for establishing a more robust infrastructure of support for teachers are underway in ways that are distinct from professional development. The hallmark initiative for this strategy was the reflective practice pilot for spring (FebruaryApril) 2011. The purpose of the Reflective Practice Pilot was to identify best 41

Connect professional support opportunities to individual need

Since the inception of the TEI proposal, one of the cornerstones of the work has been to develop a system whereby support strategies would be customized to meet the individual needs of teachers. The Professional Development and Staff Inservice (PDSI) team has facilitated much of the progress in this area. In June 2010, the MCS Board of Commissioners approved a Professional Development policy40 that requires all teachers to complete a minimum of 57 hours of professional development and reinforces the Districts commitment to time professional development to teachers individual need. The professional development requirement is parsed out into state-required in-service days (40 hours) and an additional 17 hours of minimum yearly required professional development performed online and/or during times reserved for afterschool staff activities. The professional development policy was responsive to the Districts need to address specific needs in specific areas of content and pedagogy and preceded the strategies addressed here. The District has seized the opportunity to change the character of the traditional sit and get professional development experiences. MCS has moved toward more targeted attempts to reach
39

Isabella Wilson is the first MCS Broad Fellow and Chief of Staff for the Deputy Superintendent of Academic Operations, Technology, and Innovations.
40

The MCS Professional Development policy is available at www.mcsk12.net/policy/policy.asp.

Better Support, Utilize, and Compensate Teachers practices and innovative strategies for supporting teachers practices with timely and actionable feedback. It was also designed to test scalability and ease of implementation with each strategy for delivering feedback and support to teachers. Therefore, the five components of the Reflective Practice Pilot to test mechanisms of support were decided to be: 1) written feedback, 2) co-investigation, 3) video coaching, 4) school innovation, and 5) pre-tenure support. The objectives of the Reflective Practice Pilot are: To provide feedback to one-third of teachers; To inform plans for 20112012 districtwide implementation; and To transform the mindset to focus on continuous improvement of practice. According to Ms. Banks, This could be the most critical pilot because until now, TEI has been an imaginary friend (or enemy) to teachers. This represents tangible pieces to supporting our teachers. Each component is described in detail here with particular attention given to the strategy for delivering feedback to participating teachers. It is important to note that each of these pilots makes use of emerging strategies for promoting ongoing support and development for effective teaching. The strategies and original plan of engagement described here are summarized in Exhibit #7. Written Feedback For the purposes of this discussion, the written feedback pilot is synonymous with the observation rubric field test that is described in detail in the part of the case study that outlines Strategy No.1: Define and Measure Effective Teaching. 42 Teachers (N = 600) who participated in the observation field test received written feedback after their observations. Seventy-three observers (e.g., principals, assistant principals, content specialists, and contracted observers) entered 50 schools to conduct classroom observations with the three rubrics41 that were field tested. After observations were complete, the observers delivered written feedback in the form of observation notes and brief narratives written on the actual rubrics. At the very least, teachers should know what behaviors were observed in the classroom, so this pilot tested the impact of giving teachers written feedback in the form of the actual rubric that was used to observe the classroom practice. Observers were trained on the rubrics as well as how to deliver feedback. The written feedback pilot ended March 11, 2011. At the close of the written feedback pilot, participants (teachers and observers) attended feedback sessions to discuss their experiences in the written feedback pilot. In the teachers feedback sessions, many teachers reported that their scores did not necessarily match the numerical scores that they received on the rubric. Likewise, the written feedback was less helpful if the comments were generated from the rubric verbatim. Observers stated that the written feedback was beneficial and helped teachers to behave differently in the classroom. For some reason, many of the observers were not aware of the fact that the completed rubric was given to the teachers. Interestingly, some observers stated that they were uncomfortable with having the written feedback shared with the teachers who were observed for the pilot because they were rigid and straightforward with their comments.
41

MCS field tested three rubrics for the observation rubric field test: MCS Revised Framework (current), TAP, and DC IMPACT.

Better Support, Utilize, and Compensate Teachers In other words, observers were very honest in their feedback, but admitted that they might not have been as honest in their feedback if they had known that the feedback would be shared. More specific outcomes from the observation rubric field test (or written feedback pilot) are described in detail in Strategy No. 1: Define and Measure Effective Teaching. Co-Investigation The co-investigation pilot was designed to help deepen our understanding of what happens when teachers have in-depth conversations with peer observers about what happens in their classrooms. This model was intended to provide teachers with in-person, problem-solving sessions to inform and improve practice through root-cause analysis. This method of reflective practice is quite extensive, because it involves having ongoing conversations about practice, rather than engaging in a one-time or static attempt to deliver feedback on practice. The coinvestigation requires a 1:1 ratio of feedback givers to feedback receivers. We enlisted the help of strategic partners who have worked extensively to engage teachers in this type of reflective practice. Teach for America (TFA) trained lead-teacher observers to conduct rootcause analyses in problem-solving sessions with teachers. Lead teachers were trained how to observe teachers, use the co-investigation forms, conduct a co-investigation dialogue, and move the cooperating teachers to consider and act on next steps. The TFA training session was also made available through Mediasite for other teachers to view. Four teachers from two schools42 were trained on the co-investigation model. The teachers
42

were observed twice once in-person and once by video during the pilot period, and they engaged in in-depth discussions with the lead teachers who observed them. The model of coinvestigation tested during the pilot differed slightly from Teach for Americas original method of problem-solving coaching in that the TFA model does not usually involve cameras. As a result of the collaboration during the pilot, TFA has ordered cameras and plan to incorporate aspects of video coaching into the support infrastructure that they provide to their corps members. Monica Jordan is working with TFA during the summer to develop a hybrid model for co-investigation that will include the use of cameras and adjust the strategy so that teachers can maximize use of the method without compromising the time needed to carry out other responsibilities and duties. One of the challenges for implementing the coinvestigation model was related to the amount of time needed to execute all components of the model. Teachers reported that the experience was incredibly eye-opening and transformative for their practice. However, this strategy was extremely time-consuming when the teacher pairs began to analyze the lessons. They also stated that it would be almost impossible to engage in this level of reflection in addition to their existing responsibilities in the school buildings. Because the model is so comprehensive, teachers needed to be released from teaching time to carry it out. They do not have time to prepare for the model, and time is critical to implementation of this type of reflective practice. We do not want teachers to have to give up any activities that already enrich and sharpen their skills in their schools. Analyzing the entire lessons and engaging in the coinvestigative dialogues is very time-consuming. There is more time needed to prepare for the lesson than to actually teach the lesson. In terms 43

Two schools in the Southeast Regions, Fairview Middle School and Hanley Elementary School, participated in the co-investigation pilot.

Better Support, Utilize, and Compensate Teachers of bringing this method to scale, we have to find a way to have a deep level of conversation and analysis in the current context of what happens in the classroom. Video Coaching (MET Sub Study) The purpose of the video coaching (i.e., MET sub-study) pilot was to identify the coaching strategies that have the most impact on classroom practice. The background and purpose of the MET Sub Study are described in the context of the expansion of the existing MET research project in Strategy No. 1: Define and Measure Effective Teaching. For the purposes of linking it to the Reflective Practice Pilot, the type and level of feedback delivery are the focus here. Monica Jordan has led the charge to introduce innovative technology for the purposes of reflective practice and professional growth. We received additional grant funding to conduct the sub study to investigate video-enhanced selfreflection and video-enhanced coaching as potential forms of teacher support. The MET Sub Study is designed to have coaches provide feedback to teachers through five types of reflective and coaching techniques: 1) personal reflections, 2) real-time coaching, 3) remote coaching, 4) remote real-time coaching, and 5) video-enhanced coaching. The personal reflections and real-time coaching have already gained traction with our teachers. Ms. Jordan and Dr. Kristyn Klei Borrerro43 identified lead teachers and coaches to be trained as real-time and/or remote coaches. Participants (N = 300) are MET and non-MET teachers who could benefit from the feedback and support
43

central to the methodology. All coaches are being trained on Lee Canters No-Nonsense Nurture Model and real-time coaching. The No-Nonsense Nurture Model is a four-step method that teachers can use to establish a positive classroom culture by reducing disruptive behavior and increasing time on-task for students. Teachers participated in the NoNonsense Nurture training at eight schools44 where there were at least five teacher volunteers interested in being trained. To date, approximately 300 teachers have received in-person training. Many teachers who were trained on the No- Nonsense Nurture model applied the strategies on their own with great success. Teachers admitted that they were skeptical at first but tried the strategies, and they worked immediately. The plan going forward is to conduct one school deep-dive per region (August 2011 January 2012). Upon completion of the No Nonsense Nurture training, some teachers also agreed to participate in real-time coaching. Real-time coaching involves observation of practice and feedback delivery during an actual lesson. Teachers who are being coached wear an ear bud and receive real-time feedback on their teaching from a lead teacher/coach who is standing in the back of the classroom. Nine MCS teachers have been trained to deliver real-time coaching, with two of these coaches showing exceptional promise in using the train-the-trainer model. The No Nonsense Nurture model and real-time coaching are new territory for us in terms of supporting teachers; therefore, feedback from
44

Dr. Kristyn Klei Borrerro is the Chief Program Office for the Center for Transformative Teacher Training (www.transformativeteachertraining.com).

Teachers at the following schools received No Nonsense Nurturer Training: American Way Middle, Cherokee Elementary, Craigmont Middle, Fairview Middle, Hamilton Elementary, Kirby Middle, Kirby High, and White Station Middle.

44

Better Support, Utilize, and Compensate Teachers early adopters of these methods was instrumental in our decisions to implement on a larger scale in the upcoming school year. Two hundred teachers responded to a survey about various teacher support strategies, and 83% of teachers indicated that they would be interested in participating in real-time or remote coaching. Also, the teachers were interested in discussing best practices with peers (60%), self-reflection (39%), deep-dive conversations with peers (26%), and watching/sharing videos of lessons with their peers (22%). The responses suggest that teachers are more interested in opportunities to collaborate with peers for reflective practice through coaching or information sharing. The training resonated with the people and there has been a dramatic increase in the number of those interested in being real-time coaches. Teachers reported that one of the benefits of the No Nonsense Nurture training and real-time coaching was the immediate individualized feedback. It typically takes about three sessions to see the full impact of real-time coaching, but many of the teachers who were coached in this manner experienced change instantly. There were instances where the training was met with some resistance. There was one instance where a teacher with a good track record for performance was not particularly receptive to the feedback that was given during the real-time coaching experience. Ive been teaching all of these years and no one has said anything differently [about my performance] until now. This sentiment is an indication of potential challenges related to bringing this coaching strategy to scale, especially in a context where feedback and routine performance evaluations were sparse. The momentum and impact of the No Nonsense Nurture Model and real-time coaching in our schools caught the attention of the father of these methods, Lee Canter, who wants to visit Memphis to see the impact that these strategies have had on our teachers in the short-term. School Innovation The school innovation pilot was designed to be a mini-grant protocol whereby schools sought nominal funds for the chance to build their own methods of reflective practice. The goal was to determine which practices are appropriate for school-level reflective practice as well as which practices are scalable district-wide. Schools' teams were asked to devise a program that would facilitate a culture of continuous improvement. Successful proposals would result in principals and teachers having access to District resources (e.g., cameras) and funding to test their reflective practice plans. In January, school teams convened to develop their reflective practice proposals. Fourteen school teams attended the initial working session, and 10 schools submitted a school innovation proposal for consideration. The school plans were to include the following components: 1) some form of observation practice and feedback; 2) innovative strategies; and 3) the use of technology. Once submitted, the Regional Superintendents conducted a blind review of proposals and commented on strengths and areas of improvement for each. Likewise, members of the DTTE team also reviewed the proposals and provided feedback to the school teams. Some proposals were on target for developing reflective practice plans for their schools; however, there were also proposals that missed the mark in terms of proposing to use the resources for reasons other than reflective practice. The wide range of responses to building a program for reflective practice can be attributed to a number of factors, including a misunderstanding about the request for proposal and what was meant by the term reflective practice. 45

Better Support, Utilize, and Compensate Teachers Initially, little guidance was given to teams on developing their plans, because the teams were given room to set the parameters for their work. Some school teams submitted proposals for programs that have a direct link for improving student achievement (e.g., computer labs and programs) but have little connection to encouraging teachers to engage in active and ongoing reflection of their work. The DTTE team resolved to send correspondence to all schools clarifying the definition of reflective practice and offering the opportunity for schools to submit revised plans that established moredirected plans for reflective practice. Two schools were chosen to execute their plans for school innovation. The school pilots were scheduled to begin by mid-February 2011, and end at the close of the school year; however, the schools were unable to get started due to challenges with securing the cameras for use of reflective practice. Pre-Tenure Support The purpose of the pre-tenure support pilot was to provide teachers with a quality mentoring experience to help new teachers reflect on and improve their practice in preparation for achieving tenure. The pre-tenure support pilot was completely derivative of the existing mentoring program that is provided through the Office of Professional Development and Staff In-Service (PDSI) for novice teachers. Currently, a mentor is assigned to each new teacher; protgs participate in several sessions with their mentors, and mentors are expected to document their work and activities with protgs. Although the mentoring program has been operational for years, it has received mixed reviews from teachers and principals. Teachers report that the implementation of the mentoring program is disparate across the district and is in need of evaluation. The outcomes and implementation of the mentoring programs are 46 inconclusive due in large part to the lack of clarity around the expectations of those serving as mentors and the lack of data to validate the implementation and utility of the program. In response to these concerns, the pre-tenure support pilot was an opportunity to evaluate the current programs and explore innovative strategies for supporting new teachers. The pre-tenure support pilot was intended to integrate reflective practice opportunities into a mentorship model of development. The pilot also serves as a means to shore up experiences for early career teachers so that they are oriented and developed through more focused and deliberate interactions with mentors. The reflective practice dimension of the pilot was linked with the potential use of camera technology to facilitate these interactions. It would be difficult to anticipate and differentiate the mentor experiences for the entire sample of teachers in the pilot, but at the very least, there needed to be a concerted effort from mentors to connect new teachers to the various District resources (e.g., people, departments, programs) available to help guide their early career years. Similar to the school innovation pilot, the pretenure support pilot made little headway due to the challenges in securing cameras. We learned important lessons from the spring 2011 Reflective Practice Pilot. The successes and challenges from the individual pilots provided great insight for our broader approach to individualizing support for teachers. A onesize-fits-all model is not appropriate for a district of 7,000 teachers as evidenced by the various opportunities for reflection made available to teachers. Moreover, most of the 800 teachers who participated responded favorably to the variety of opportunities made available through the Reflective Practice Pilot. Likewise, we learned that teachers are most interested in

Better Support, Utilize, and Compensate Teachers reflective practice that allows them to engage others in conversation about their teaching. As stated, we struggled with several aspects of reflective practice implementation, not the least of which was the acquisition and deployment of cameras. Many schools already had the camera equipment, but the ordering of new cameras was delayed and stalled the pilots that relied on the use of cameras. The coordination of efforts across multiple departments and organizations that already provide some type of support teachers was also a challenge. Many District departments and partners have programs and services that touch pre-tenure teachers. For example, new teachers who are affiliated with our partner programs (e.g., Teach for America, Memphis Teacher Residency, and Memphis Teaching Fellows) receive intensive support as a part of their commitment to their respective programs. It behooves us to understand better how these and other strategies could benefit teachers in a way that is not duplicative and has the greatest reach across the district. The information collected through each pilot was very instructive in our efforts to build a program of support that is responsive to the collective and individual needs of all teachers. Initially the goal was to identify the single most impactful strategy for the District; however, it was equally important to identify the factors that would facilitate and impede the development of stated program. Create new and differentiated career paths The Reflective Practice Pilot presented opportunities to see what differentiated roles and responsibilities for teachers might look like in the coming years. In the co-investigation and video coaching strategies, the peer-coaching and support responsibilities of the more experienced teachers most resemble possible roles and 47 responsibilities of lead, professional, or master teachers. Granted, the specific criteria for the differentiated career paths are yet to be determined, but feedback and outcomes from these pilots will inform future design and implementation of this particular initiative. We have adjusted the timeline for piloting the differentiated career paths to coincide with full implementation of the TEM (20112012). The Coordinator of Teacher Career Management, Mike Neal, will oversee this dimension of the work.

Implement a new base compensation structure

. . . if in a year from now, the dialogue, conversation, and support for providing teachers a livable wage based on effectiveness would be an acceptable reality. Irving Hamer When asked what successes he would like to see as a consequence of the teacher effectiveness reform, Deputy Superintendent Hamer responded with the foreshadowing of the most groundbreaking aspect of our body of work, the new base compensation structure. It is no secret that teachers are underpaid and underappreciated across the nation. Deputy Superintendent Hamer adds, The current system diminishes the craft. We are building a system that is responsive to the [teaching] profession. We also believe that we need a contemporary compensation system to recruit and retain good teachers. The new base compensation structure was originally scheduled to follow the development of the new evaluation system and roll-out during the 20132014 school year. Because Tennessee law now requires every teacher to be evaluated every school year, the development of our new evaluation system (i.e., Teacher Effectiveness Measure or TEM) was accelerated to be aligned

Better Support, Utilize, and Compensate Teachers with the law. Moreover, the timelines for the implementation of strategic initiatives driven by the TEM reflect earlier launch dates. While the work of building a new compensation structure is listed here as a milestone for the larger strategy of better supporting, utilizing, and compensating teachers, it is by no means ancillary to the MCS model of teacher effectiveness reform. According to Deputy Superintendent Hamer, a new base compensation structure is a critical strategy for the teacher effectiveness reform. His sentiment is simply: This is the largest body of work that we have to do; this is the work! This is where the heavy lift has to take place. We can do everything else, but changing the way we compensate teachers is cutting-edge work and is exactly the human capital management that is TEI. It is not uncommon for school districts to layer on financial incentives to an existing salary, but it is totally unprecedented to challenge a statusquo system for compensating teachers that has been in place for nearly a century. We acknowledge the research that refutes a positive impact of incentive pay on performance and development of employees. We are not looking simply to enhance our current practice of paying teachers extra money here and there with little consequence for their performance; we do not want to compensate ineffectiveness. Rather, we intend to build a base compensation system that aligns to differentiated career paths for teachers and rewards teachers for effectiveness. At this juncture, we have the opportunity to execute a complete overhaul to the MCS compensation structure knowing that it will demand a philosophical, conceptual, and cultural shift in the way we think about the teaching profession. Collaborative work plans are in development with several expert compensation teams. We have enlisted the help of Dr. Matt Springer, Rick 48 Lantz, and Service Master. The burgeoning relationships with our expert teams are a product of conversations and commitments through the TEI Advisory Board, a body of community representatives from business, philanthropic, and educational organizations that support the teacher effectiveness reform of MCS. More specifically, connections with Service Master and Rick Lantz were made through the Advisory Board seats held by Memphis Tomorrow and International Paper, respectively45. The District has issued an RFP to identify a consulting firm who will also help to design the new compensation structure. With industry leaders and experts at the table and ready to work, the progress of this initiative has surpassed initial expectations. In early meetings, partner teams and District personnel brainstormed ideas for building the new system in a way that challenges the status quo and changes public policy at the local and state levels. The current practices for compensation and exchanging information for ideas for the work ahead were a good starting point for discussion and big-picture ideas. For example, the residual impact of the Teacher Career Ladder Program46 that was in its prime during the 1990s may be a good reference for forthcoming compensation plans. In the Career Ladder Program, teachers were compensated based on certifi45

Blair Taylor, President of Memphis Tomorrow, and Kim Wirth, Executive Director for International Paper Company are members of the TEI Advisory Board. A complete listing of the TEI Advisory Board members is available at http://www.mcsk12.net/tei/board.asp.
46

The Teacher Career Ladder Program is described in detail in The Teacher Effectiveness Initiative Case Study 2010: Strategy No.3: Better Support, Utilize, and Compensate Teachers available at www.mcsk12.net/tei.

Better Support, Utilize, and Compensate Teachers cation and years of teaching experience within five levels of career advancement. There are still MCS teachers whose salaries are driven by the Career Ladder Program. It is important that we engage these teachers to help understand the circumstances that led to the demise of the Career Ladder Program as well as navigate the challenges that may emerge with a new system. We plan to institute the new compensation with the August 2012 class of new teachers coming to the district. We are now in the process of identifying a middle school site to test the strategic staffing methods.

Cluster high-potential teacher recruits in schools with the most high-need students
The progress for this initiative is described in Strategy No. 2: Make Better Decisions about Who Teaches. The STARS team has given priority to teachers from partner programs that seek to have clusters of their teachers working in the same school or at least the same feeder pattern. We are committed to making sure that the clustering strategies are successful for partner programs, because clustering is a unique support strategy for candidates who are more likely to be placed in high-priority schools with high concentrations of high-needs students. In the coming years, this initiative will be consolidated with the placement strategies that appear in Strategy No. 2: Make Better Decisions about Who Teaches.

Strategically place the best teachers where they are needed most
Increasingly, we are seeing the need to staff strategically the schools that are in need of more high-performing teachers. Coleman Elementary school has been approved as a model for strategic staffing methods in 20112012. Prior to the end of the 20102011 school year, the principal of Coleman Elementary approached District leaders in a plea for her school. She mentioned that she had seen the student growth data for her school and noticed that she had only a handful of high-performing teachers (i.e., levels four and five according to value-added measures). Specifically, there were a couple of teachers who fell into levels three and four, but none were classified as level five. The rest of the teachers were classified as level one. This situation naturally occurred and has served as the impetus for the launch of a Strategic Staffing Subcommittee. Although no concrete decisions have been made to date, the possibility for strategic planning and placement of highperforming teachers in this case is exactly what we intend to do for schools across the District. This kind of strategic staffing has implications for our evaluation and support initiatives. For example, with the TEM in place, we will be able to identify additional opportunities for placing our best teachers where they are needed most. 49

Build a service oriented culture in the district toward teachers

This is the first year that the milestone to build a service-oriented culture for teachers in the District appears in the plans. In a February 2011 in-service survey, teachers responded to the open-ended question, What can MCS do to better support teachers? Teachers provided various responses, including improved professsional development and increased peer collaboration. The majority of teachers replied that they would feel more supported if the paperwork was reduced, and they had more time to teach (see Exhibit #8). Ms. Jordan is collaborating with a teacher working group to identify noninstructional time commitments of teachers and to make recommendations to streamline their workloads. Beginning late summer (or early fall) 2011, Ms. Jordan was set to convene teacher

Better Support, Utilize, and Compensate Teachers groups and administrators to prioritize noninstructional demands based on feedback from various stakeholder groups. Together, these groups will determine strategies to ease the noninstructional time commitment and workloads of teachers. a specialized and differentiated program of support. The workings of a new base-compensation system and differentiated career paths for teachers are not fully defined at this time; however, working groups and strategic partner teams are identifying best practices and strategies that are appropriate for the District.

Conclusion
In preparation for the upcoming school year, we are finalizing our work for a new framework for teaching and learning. We have partnered with Insight Education Group, the developers of our new observation rubric, to reshape our understanding of what it means to measure, monitor, and manage effective teaching. Inherent in this new framework is a commitment to support teachers in ways that are new and different from the traditional methods. In the broader landscape of our work, teacher support refers to an infrastructure that facilitates the continual learning, differentiated responsibility, and deliberate enhancement of teaching as a profession. With this, the development and implementation of a comprehensive program of support for teachers has to be as dynamic as the career development of teachers that we envision. The strategies of better supporting teachers have improved in major ways due in part to the launch of the Reflective Practice Pilot. Collectively, the pilots engaged 800 teachers across the District. It is not clear whether one type of reflective practice will be the banner of support for MCS, nor is there anything that precludes the District from employing some version of all of the reflective practice strategies in the future. We continue to endeavor to supply teachers with timely, individualized feedback. We want to be intentional in prescribing support, particularly for those teacher groups47 requiring

Ongoing Issues and Next Steps

The most salient challenge for this strategy is the use of video cameras to capture teacher practice. There is a complicated matrix of issues related to the use of video cameras that continues to wage concern in the minds of teachers. For the purposes of the pilots, there seems to be some agreement with the teachers union such that participation in pilots with video cameras is strictly voluntary. At beginning of the Reflective Practice pilot, Monica Jordan launched a camera tour to increase exposure to the use of camera technology for professional growth and collaboration (December 2010January 2011). The camera tour was set up so visitors could touch the cameras and examine their operations. Further, Ms. Jordan invited the visitors to and share early thoughts about the use of cameras in classrooms. She created a catalogue of potential barriers and exciting ways to use the technology based on the responses and feedback of teachers who participated in the camera tour. Many of the teachers volunteered to participate in reflective practice pilots as a result of their experiences during the camera tour. There are clear District policy implications for the use of video cameras. Further, the decisions

47

Pre-K through 3 grade, middle school, and pretenure teachers have been identified as those who

rd

are particularly vulnerable and in need of intensive, specialized support programs.

50

Better Support, Utilize, and Compensate Teachers made at this point have set early precedence for the use of video cameras across the district going forward. Teachers privacy options were among the most debated topics with regard to the new practice and policy of video observation. Privacy of video content was also a controversial issue in that teachers were generally concerned about the punitive use of video content for high-stakes decision-making, specifically evaluation. The PDSI office also drafted a set of protocols for the use of video cameras. Per these interim protocols, teachers have complete ownership of their videos, thereby having complete decision-making authority to share the content with other colleagues or delete the video content at their discretion. The District needs solid guidance on how to proceed with the use of video cameras in academic operations (e.g., observation, professsional development, evaluation). A policy for video capture and video use is currently under development.

51

Better Support, Utilize, and Compensate Teachers

Exhibit #7:

Spring 2011 Reflective Practice Pilots

Spring 2011 Reflective Practice Pilots

MCS Reflective Practice

OBJECTIVES 1. Involve 1/2 of all MCS schools 2. Inform plans for 2011-2012 district-wide rollout 3. Transform mindset to focus on continuous improvement
1

Written Feedback

Co-Investigation
Provide teachers in the observation rubric II study with in person problem solving sessions, utilizing camera resources as needed
Teachers: 60 Schools: 10
Feedback givers: High-performing peers and content specialists using video observations

Video Coaching
Provide teachers in the MET substudy with access to real-time and post observation coaching, utilizing camera resources as needed Teachers: 100 Schools: 14
Feedback givers: Teacher-selected and assigned coaches

School Innovation

Pre-Tenure Support
Provide pre-tenure teachers with quality mentorship program to help new teachers reflect on and improve their practice
Teachers: 140 Schools: 12
Provided by: Mentors in MCS Teacher Induction Program

Strategy

Provide teachers in the observation rubric I study with written feedback

Provide schools with the opportunity to build their own methods of reflective practice

Feedback Participants

Teachers: 610 Schools: 61

Feedback givers: Principals, APs, hired observers and content specialists

Teachers: Schools: 2
Provided by: Varies by school

REFLECTIVE PRACTICE FOUNDATION Partnership with Professional Development Partnership with School Operations

52

Better Support, Utilize, and Compensate Teachers

Exhibit #8:

2011 Survey Results on Teacher Support

2011 Survey Results on Teacher Support

Open-Ended Reponses: What can MCS do to better support teachers?
20%

15%

10%

5%

0%

53

Improve the Surrounding Context to Forster Effective Teaching

Strategy No. 4: Improve the Surrounding Context to Foster Effective Teaching

of our strategic partners, The Efficacy Institute,48 recruited, trained, and deployed approximately 170 student envoys and 100 teacher envoys in schools (elementary, middle, and high school) across the district. The Efficacy team worked closely with the MCS Division of Student Support Services to establish school teams and activities related for the Envoy Project. The team in Student Support Services has also been able to identify and link student and school-level data to the Envoy Project and other school-level attempts to improve the school culture and climate. MCS has several available data sources (e.g., PBIS49, Teacher Working Conditions, Tripod and Safety) that inform our understanding of school climate and culture. In some cases, these data have had little impact on decisions made to address the concerns of different stakeholder groups. Disparities in the approach to data collection (e.g., prevention vs. intervention), reporting requirements (e.g., types of incidents), and interpretation or use lead to a distorted view of what the context for teaching and learning is in all of the schools. There is a need to crossreference all available data to generate schooland district-level reports and to develop appropriate plans of improvement. We have not yet established an integrated technology platform that facilitates timely and accurate data storage, retrieval, and reporting. Last year, the District started down the path to an integrated technology system with an interim
48

Teachers are effective to the extent that they can teach in settings and under circumstances that facilitate positive teaching and learning experiences for themselves and their students. Principal leadership capacity, school culture and climate, and technology are central areas of focus in our efforts to create conditions that would allow teachers to impart meaningful educational experiences to their students consistently.

Context
The school principal is one of the most influential factors to the progress of the TEI. We understand now more than ever the importance of developing strong leaders who buy in, execute the reform, and have the capacity to empower their teachers to do the same. We instituted the Leadership Effectiveness Initiative (LEI) to increase principal engagement with respect to the TEI. By design, the leadership effectiveness work is an analog to the teacher effectiveness reform in that it seeks to improve the evaluation, training, and retention of effective principals. Strategies to improve school culture and climate yielded early successes through the implementation of the Envoy Project. During Year 1, one 54

The Efficacy Institute www.efficacy.org.

49

Positive Behavioral Intervention and Support www.pbis.org.

Improve the Surrounding Context to Forster Effective Teaching technology solution for data reporting and an electronic evaluation system. The interim technology solution was made available to report on data requests and outputs for the October 2010 stock-take with the Bill & Melinda Gates Foundation, but it is not enough to maintain ongoing analyses and reporting requirements. The strategic initiatives from Year 1 remain the same, but the focus has shifted in some areas. Accordingly, this case study presents the planning and progress that have taken place to date. The Year-2 goals for improving the context for teaching and learning are as follows: Improve principal leadership capacity Improve school culture to create conditions that foster effective teaching and learning Develop a new technology platform that will support data-driven decisions In order for us to reach our goal of district-wide effectiveness, we need principals to be instructional leaders, not just building managers. District leaders have ascertained that 60% 70% of principals time was dedicated to operational issues (e.g., cafeteria duty, building maintenance) not instruction. District leaders are trying to determine how reduce to the operational responsibilities of principals and empower them to maintain laser-like focus on instruction and effectiveness. To facilitate principals refocus on instruction in their buildings, we intend to develop and implement training tools to support principals in delivering more robust evaluations, providing consistent feedback, and referring teachers to appropriate resources for support and development. Specifically, we have begun to develop a comprehensive principal syllabus with an emphasis on effectiveness for their monthly training sessions. Institutes. In particular, the Forum for Innovative Leadership has operated as the district-wide principals annual summer training experience. The LEI is intended to work as the analog to the TEI and operate as a comprehensive structure of support and development for school and District leadership. However, this initiative has not yet reached a level of implementation comparable to that of the TEI. Recently, New Leaders for New Schools (NLNS)50 has been officially commissioned to manage the LEI body of work and move it forward toward promoting school leadership effectiveness.

Implementation & Findings Principal Leadership Capacity

The scope of leadership effectiveness centers on cultivating strong leadership in schools across the District. Moreover, our efforts and experiences to date reinforce the need for more buyin and execution on the part of our school-level leaders. In the past three years, there have been 100 changes in school leadership. Therefore, it is imperative that we maintain a strong, consistent pipeline for high potential principals to take leadership in our schools. Since 2009, the Urban Education Center (UEC) has served as the primary preparatory programs and pipeline for school leadership in MCS. The UEC continues to host annual training and development opportunities for school leaders, namely the Forum for Innovative Leadership, Executive Leadership Program, and Summer 55

50

New Leaders for New Schools www.nlns.org.

Improve the Surrounding Context to Forster Effective Teaching

Improve school culture and climate

The student-centered approach to creating positive school cultures, the Envoy Project, continues to be a distinguishing factor in our work to improve the context for teaching and learning in school buildings. In November 2010, nearly 600 student envoys from all grade levels attended a fall conference to prepare them to be change agents in their schools. To date, there are approximately 800 student envoys in schools throughout the district and 150 teacher envoys trained in the efficacy model. Student envoys were trained to take responsibility for their own achievement and school environment and to demonstrate those values throughout the school day. The first cohort of middle school student envoys who were trained during the summer 2010 Leadership Camp has already begun to demonstrate belief in and commitment to efficacy. Student envoys delivered efficacy messages to members of the student body during morning announcements, displayed promotions in the form of posters and banners around the school building, and gave a State-of-theSchool address to inform students about the conditions in their school. Representatives from the Bill & Melinda Gates Foundation had the opportunity to visit Hamilton Middle School to sit in on an efficacy presentation by their student envoys. Colleen Oliver, our former program officer with the Foundation, had this to say about her experience at the school: The students were very articulate, insightful and wise about what needs to occur to change the culture of their school. They articulated how hard it can be to serve as a leader around doing the right thing but it was impressive that they were willing to serve in this role. They were genuinely proud that the school year is off to a good start with no fights, etc. I was impressed with the new principal. He has only been there a 56

month but it was clear that he is committed to changing the culture, and I think he's already put in place some good things; the envoy teachers were also very committed to this work. Additional recruitment and training opportunities for student and teacher envoys were expanded with a MCS Envoy Youth Conference in March 2011, and the 2011 Summer Leadership Camp, a five-day experience for incoming envoys. The Efficacy Institute and MCS Student Support Services are working to bring 50 new schools on board for the 20112012 school year. Student envoys have earned recognition and support across the District and most recently on a national level. One of the Envoy Projects most notable laurels for this year came in the form of an invitation from Alberto Retana, the former Director of Community Outreach for the US Department of Education, to have MCS student envoys attend the United States Department of Educations (USDOE) Voices in Action, National Youth Summit in Washington, D.C., to discuss the state of education across the country. The invitation came as a result of conversations between Mr. Retana and student envoys during his visit to the district-wide ThinkShow!51 in early November 2010. During the session, which was a scheduled event as a part of the USDOE National Youth Listening Tour, students had the opportunity to share their experiences as envoys with Mr. Retana and other visitors. The students grasp of efficacy principles and vested interest in
51

ThinkShow! is the MCS district-wide, projectbased, exhibition whereby students demonstrate learning beyond standardized testing. The highlight of ThinkShow! is the participation of community members (~7,000) who visit schools to judge student work. Additional information is available at www.thinkshow.org.

Improve the Surrounding Context to Forster Effective Teaching their educational experiences compelled Director Retana to extend the invitation to attend the National Youth Summit. Director of School Services and Training with the Efficacy Institute, Barbara Logan, shared, This is something they will never forget. We want them to be able to share this [experience] with their classmates and friends. MCS had the largest representation at the conference with 46 students from 26 schools. During the conference, students heard from national leaders in politics and education. They also interacted with other students from across the nation to talk about their roles in improving the state of education. I accompanied the student envoys on their journey to the National Youth Summit in Washington, D.C., to document their experiences with other ambassadors for education. After having spent the weekend with the student envoys and witnessing their participation in the National Youth Summit, it occurred to me that of all of the stakeholders we seek to inform and engage in our reform efforts, we have not necessarily done our due diligence with the group who are our primary stakeholder group of interest our students. During the summit, students spoke about wanting to have input into their teachers evaluations, engaging school district leaders, and making a difference in their schools. Although the Envoy Project has been successful as a vehicle to mobilize students to take responsibility for their education, we have to be more intentional about our efforts to empower students and to inform students about the reform that surrounds them. In March 2011, the Envoy Project hosted the first Envoy Spring Conference. Students had the opportunity to share their work with peers, teachers, principals, and District leaders. Likewise, they took advantage of the opportunity to speak directly with District leadership. The students addressed a panel of District leaders 57 and asked specific questions about the Envoy Project and their schools. Further, students welcomed the leaders openness and advice throughout the conference. The Envoy Project continues expansion across the District. In addition to the recruitment of more student and teacher envoys, the program promises to position students for more exposure and leadership opportunities. One such example of increased exposure is the result of students experiences with documenting their trip to the National Youth Summit. Student envoys who participated in the technology and media classes are working to create an envoy television show. The show will feature highlights of the program (e.g., National Youth Summit) as well as student presentations of efficacy principles. There is a wealth of available information to enlighten decisions about improving the culture and climate of schools. We have PBIS, Teacher Working Conditions, Tripod, and Safety data at our disposal. This year marked the resurgence of the PBIS program in the District. In conjuncttion with the PBIS benchmarking tool, security data is valuable in understanding school vulnerabilities for specific incidents (e.g., fights, weapons). The Tripod and Teacher Working Conditions surveys provide insight into school climate and culture through the lenses of those in particular schools. While the Tripod survey allows students to provide feedback on aspects of specific classroom cultures, the Teacher Working Conditions survey allows teachers to provide feedback on various dimensions of the school culture and climate. The data from last years administration of the Teacher Working Conditions survey has already been used to identify schools within the district that require immediate attention to issues of school climate, including limited enforcement of the student code of

Improve the Surrounding Context to Forster Effective Teaching conduct and marginal support from the school administrator. This year, instead of the Teacher Working Conditions survey, a school climate and culture survey, TELL Tennessee52, was piloted to capture the perceptions of all schoolbased licensed educators about learning conditions, through an anonymous process. The District encouraged all teachers to complete the online survey so that any data shared with schools and districts state-wide would also be representative of MCS. Approximately78% of MCS teachers completed the survey, and schooland district-level reports are available at www.telltennessee.com. The key to staying proactive in the approach to creating optimal learning environments is to develop a system whereby all of the available data sources are cross-referenced and result in not only a comprehensive view of school conditions but also an equally comprehensive plan to address issues that occur in the school which might otherwise jeopardize the educational process. Toward this end, we charged the Culture and Climate task force with the responsibility of exploring the data and making recommendations for system-wide efforts to create positive learning environments. Going forward, we intend to equip District leaders and school principals with the data and resources to develop and implement school-wide improvement plans. evaluations and school data analyses. RANDA Solutions has designed the Online Principal and Teacher Evaluation System (OPTES) which is the electronic interface of the current framework for evaluation. This electronic tool is vital to our capacity to manage and conduct nearly 30,000 observations district-wide. Although RANDA Solutions has developed what is needed to maintain the evaluation tool itself, we are exploring options for having a platform that connects the evaluation tool (i.e., TEM) to the rest of the evaluation process (i.e., recommendations for professional development and support). Tableau53 is the program of choice for providing school and District leaders with an array of realtime data about the schools. Principals will be able to have the data at their fingertips as they walk through their school buildings. It is a valuable tool to help principals and District leaders to monitor student-, teacher-, and schoollevel data. The software also helps facilitate cross-analytics on various data points that relate to performance in the building (e.g., attendance, discipline, proficiency levels). The Tableau software will be piloted with schools in the Striving Schools Zone. The goal of pilot is to equip 5565 school principals with these tools during the coming school year. The electronic vacancy management form that was developed by the STARS team also represents progress toward the move toward more streamlined process management and implementation through electronic tools. We intend to expand this utility of this and other online resources that lead to more timely interfaces with stakeholders and archive relevant data for continuous improvement.
53

Develop a new technology platform

Technology is a foundational component of all teacher effectiveness strategies. We have made some progress in identifying technology solutions for our most immediate needs, namely
52

Teaching, Empowering, Leading, and Learning (TELL) Tennessee will be administered spring 2011 and spring 2013. Additional information is available at www.telltennessee.com.

Tableau www.tableausoftware.com

58

Improve the Surrounding Context to Forster Effective Teaching We continue to work toward securing a technology platform that serves all of the functions needed to manage our comprehensive, human capital strategy, particularly for forthcoming development in areas related to staffing, compensation, and the like. The IT department completed an audit to identify long-term technology needs and solutions.

Ongoing Issues and Next Steps

There continue to be challenges in implementtation of the three-pronged approach to improving the surrounding context to foster effective teaching. Among the most prevalent challenges related to this strategy are Stakeholder buy-in and internal capacity. Students are effective change agents in their schools to the extent that the culture in which they find themselves is conducive to change. The key is to ensure that all stakeholders understand that efficacy is a belief system rather than a matter of compliance. Recruitment and training of many students and teachers for the Envoy Project continue to be quite successful; however, it has been difficult to foster the same level of success in getting teachers and principals to buy in to the Efficacy mindset and embrace it for their schools. Without a network of educators and leaders to support high expectations of achievement in the schools, the students remain at a disadvantage with respect to their own academic outcomes. The following is an account shared by a parent who is concerned about an efficacy dilemma faced by her son. Her son, a student envoy, was in a class with a teacher who did not share his efficacy mindset: My son wanted to exercise the efficacy principles in one of his classes. He took a Geometry test and wanted to retake the test to make a better grade. The teachers policy was that you could retake the test only if you failed. He didnt fail the test [with an F], but he failed by his standards. He asked her over and over again if he could take the test again, but she wouldnt let him because he didnt fail. He said, So because I didnt fail, I dont have the opportunity to do better? The teacher stuck to her policy. 59

Conclusion
Improving the culture and climate of schools across the district remains a priority. We have maintained our position that creating and sustaining an environment that promotes effective teaching and learning cannot be achieved through a one-dimensional approach. Whereas this strategy was previously focused on gearing up the student-centered approach to cultural shifts in schools, Year 2 involved enhanced efforts to incorporate other factors that contribute to schools culture and climate. The Envoy Project is gaining momentum and has poised our students to take ownership and leadership in their educational experiences. The Efficacy Institute has been charged with identifying key academic, social, and emotional variables that reflect the programs impact on students, teachers, and schools. In like manner, we have to be more intentional about using school-level data to drive decision making and coordinate efforts to improve the climate and culture for schools across the District. Although we continue to rely on separate technology solutions for different aspects of the work, we have identified the issues that require urgent attention and have begun the search for a fully integrated ERP system. To date, an RFP has been issued but not filled for a project management solution to precede the implementation of the long-term ERP solution.

Improve the Surrounding Context to Forster Effective Teaching This student wanted to try again and do better because he had high expectations for his own achievement. The question really is: how do we support a demand student who is in a classroom or school building that does not ascribe to the efficacy model? Our internal capacity to manage the culture and climate work has been met with some challenges. For instance, Mr. James Bacchus, former MCS Chief of Student Support Service, led the school culture and climate body of work but took a position outside of the school district. Mr. Wayne Booker (former Academic Coordinator for the Southeast Regional Office) took over that position in July 2011 after an unsettled period of interim leader-ship. He works closely with Marqui Fifer of the Department of Teacher Talent and Effectiveness, who chairs the Culture and Climate Task Force.

60

Other Strategies Related to Success of TEI

Other Strategies Related to the Success of TEI

requires that the results from evaluations be used in employment decisions including, but not necessarily limited to, promotion, retention, termination, compensation, and attainment of tenure status. Local districts are also given the autonomy to develop and gain state approval to use a compensation system different from the state. In support of the reform, MCS is engaged in policy and legislative activities that support the goal of having an effective teacher in every classroom, every day. Since inception, the teacher effectiveness reform has garnered the support of a wide range of community stakeholders. In fact, MCS formed a TEI Advisory Board, comprised of local business, philanthropic, and higher education organizations, to guide the Districts work and to serve as a collective body of advocates for the sustainability of the work. Additionally, in Year One, we successfully established relationships with prominent community organizations to begin the work of galvanizing support for ongoing reform. Historically, MCS has struggled with maintaining effective and ongoing communication strategies with various internal and external stakeholders. Despite the Districts commitment to reform and the benefit thereof, negative perceptions about the work of the school district outweigh the positive stories of achievement and triumph of the many MCS principals, teachers, and students. Moreover, MCS personnel admit that they have reform or initiative fatigue and believe the TEI is another fleeting reform that will subside when leadership changes or new reform ideas occur. Based on recent polls in the Memphis community, we are faced with the formidable tasks of building trust among 61

Context
For Memphis City Schools (MCS), policy reform has been the most appropriate means of ensuring the sustainability of reform efforts. Early on, the District sought to establish protocols and procedures that would keep the reform efforts on track despite changes in district leadership, personnel, and priorities. One such example is the policy work that began in 2008 with the Cradle to Career reform agenda, the Districts approach to impacting the lives of youth at critical junctures in their education and development. The TEI is one dimension of the Cradle to Career reform agenda. Shortly after Superintendent Cashs arrival to Memphis, he and the MCS Board of Commissioners worked together to align the Cradle to Career body of work with appropriate academic, personnel, and operational policies. The implementation of reform strategies as well as the transformation of district policies to institutionalize reform preceded much of the policy and legislative changes at the State level. In 2010, the Tennessee General Assembly enacted the Tennessee First to the Top Act that requires annual teacher and principal evaluations based, in part, on student growth data and classroom observations. The legislation also

Other Strategies Related to Success of TEI stakeholders, dispelling myths around the TEI strategies, and building widespread support to move our reform agenda forward. The Tennessee First to the Top Act requires annual evaluation of all teachers. Further, the legislation enabled school districts to make employment decisions, such as promotion, retention, termination, compensation, and tenure, based on teacher evaluation and student achievement data. In terms of the observation component of teacher evaluation, MCS is exploring the use of video cameras to facilitate evaluations as well as to support teachers. MCS teacher evaluation policy was developed to establish protocols around the use of TVAAS data for evaluations. Likewise, revisions to the evaluation policy must now incorporate the appropriate use of video technology. In 2011, a new tenure law was passed to extend the probationary period for teachers to receive tenure from three years to five years. The law also makes a probationary teachers tenureeligible status contingent upon receipt of an overall performance effectiveness rating of above or significantly above expectations on teacher evaluations. Accordingly, MCS policy must include the process for receiving tenure and what, if any, additional criteria above and beyond the evaluation might be used to aware tenure. Although the professional development policy went into effect about a year ago, there is a need to address other meaningful professional experiences in a policy for teacher support. For example, the District is working to create a peerassistance and review (PAR) program. The goals, roles, and responsibilities associated with the implementation of this type of teacher support opportunity need to be developed in alignment with a teacher support policy. This is particularly important as the implementation of a PAR program also has implications for evaluation and possibly tenure attainment.

Implementation & Findings Influence and track policies to support TEI

Policy development at the district, state, and federal levels is critical to entering into and sustaining education reform. According to Natalie McKinney, the Director of Policy and Legislative Services, Policy work involves a continuous process of assessing the effectiveness and sustainability of policies created, as an accountability measure, to ensure that the districts goals are being appropriately addressed. This necessarily includes evaluating district policies in terms of the changing and evolving district goals and expectations; the ever-changing district parameters; new and relevant research; changes in district capacity; and the volatility of stakeholder input and impact. Ms. McKinney is referring to the reoccurring themes considered when developing or revising policy in the interest of reform. To date, MCS has developed or revised the following policies to support its efforts around effective teaching: 1) Teacher Effectiveness; 2) Teacher Effect Data; 3) Professional Development; and 4) Effective School Leadership. The development of policies related to teacher evaluation, teacher tenure, and video capture are still underway and possible revisions are proposed in the areas of teacher effect data, teacher support, and teacher compensation. State legislative changes have made it possible for district policy change. The discussion here will focus on the relationship of State policy to MCS policy in the aforementioned areas of implementation.

62

Other Strategies Related to Success of TEI The current law allows school districts to develop alternative compensation systems. MCS is beginning the work of developing a new base compensation schedule for teachers. We have to be thoughtful in the way that we design the Currently, the Tennessee House and the Tennessee Senate have proposed different revisions to the Education Professional Act regarding collective bargaining. Whereas the Senate version completely eliminates collective bargaining, the House redefines collective bargaining. Once both sides reach a compromise, MCS will need to develop policies for areas restricted from collective bargaining, including but not limited to working conditions, transfer, differentiated pay plans and compensation, layoffs, and seniority. public fundraising campaign was delayed because it would coincide with the political debate and uncertainty around the surrender of the MCS charter. After several months, Diane Terrell, a board member for the MCS Foundation, agreed to work as a consultant for TEI strategic communications. Ms. Terrell is leading the work around communication strategies to support the MCS Foundations public fundraising campaign. With the consolidation of schools imminent, MCS acknowledged the opportunity to use the public campaign as a vehicle to solidify and signal the entire communitys commitment to teacher effectiveness reform. Accordingly, the I Teach. I Am was designed to be a teacherfocused strategy to increase engagement of all stakeholders. Ms. Terrell contends that the I Teach. I Am campaign humanizes and personalizes the reforms that are underway in Memphis City Schools. This communications strategy is multifaceted with emphasis in four particular areas: 1) in-school recognition, 2) public space promotion, 3) grassroots engagement, and 4) social and digital media connection. The major touch points for teachers and the I Teach. I Am campaign manifest through the inschool recognition component of the campaign. Each month, school-level personnel will nominate teachers to be the TEM Professional Teacher Award winner. The monthly recognition events will be based on teacher performance data and culminate into the TEM Prestige Awards ceremony (formerly the Prestige Awards). Teachers are being featured in public spaces, including print ads, movie theater PSAs, buses,

Develop communications strategy around TEI

Last year, MCS issued an RFP to identify partners who could help with developing and implementing a strategic communications plan around the TEI. Although the District has always had the Department of Communications to handle any communications, media relations and marketing for all of MCS, it was important to identify partners to help focus specifically on the communications surrounding the teacher effectiveness reform. After review, three communications firms were chosen to move the strategic communications of the TEI forward Reingold (Washington, D.C.), Red Deluxe (Memphis, TN), and Trust Marketing & Communications (Memphis, TN). Despite the new partnership matrix, the communications work was slow to start and gain traction. Initially, the public fundraising campaign was expected to follow the tremendous success of the private fundraising campaign that yielded nearly $20 million in a matter of months. However, the 63

Other Strategies Related to Success of TEI bus transits, billboards, and the new website54. There is power in seeing and viewing a public recognition of the impact of teachers on our students and our community. This campaign allows teachers to share their stories with the community. It is one thing to have teachers proud to teach in Memphis, but it is equally noteworthy to have a community join our efforts to uplift the profession of teaching. The I Teach. I Am public campaign launched at the second annual Practitioners Summit in August 2011. The campaign collateral included lifesized posters that featured teachers from schools across the district as well as a backdrop of all MCS teachers names in print. Likewise, videos of teachers speaking about their commitment to educating children were played on the wide screens during the opening sessions each day of the conference. Teachers were excited to see their names and pictures of themselves and their colleagues sharing their teaching philosophy and making public commitments to the children of Memphis. The grassroots engagement portion of the campaign is intended to mobilize stakeholders, particularly the business community, to donate their time or money to the success of our schools. The first initiative derived from coordinated efforts to bolster the Adopt-aSchool program and improve school culture and climate. The Touch a Teacher Lounge is gaining momentum as schools work with their adopters to remodel teachers lounges. The social and digital media component of the campaign is meant to be the intersection of the in-school recognition, public space promotion, and grassroots engagement. In addition to the I Teach. I Am website, www.mcstei.com was also created to raise public awareness of what is
54

happening in MCS and to promote continuous engagement. For examples, school district personnel and partners blog about educational issues, thereby creating a forum for transparent and relevant conversations about reform issues. Teachers stories and elements of the public campaign all land on the website as well. The goal is to have the public engaged in a meaningful way on a regular basis. For the coming year, the I Teach. I Am campaign promises to represent our framework for communication and engagement. The current strategies will be expanded to touch more stakeholders and increase our capacity to foster support for reform.

Build community advocacy around TEI

To date, we have taken great pride in the communitys demonstration of support for the Districts reform efforts. The TEI Advisory Board continues to serve as the collective representation of community support for the work. Stand for Children is one of our community partners who has been instrumental in building a groundswell of support for ongoing reform among constituents in the city of Memphis. Stand for Children is a national organization whose work is to build community advocacy for public education. The work of Stand for Children across the nation focuses primarily on building capacity for parents and teachers to become advocates and change agents in the community through education and training, public policy, and endorsement of elected officials. The Memphis Chapter of Stand for Children has organized several opportunities to educate and engage others in public education. They are spearheading efforts to establish a strong advocacy base through team building, the TEI community education coalition, and campaigns around local education issues. For 64

I Teach. I Am (www.iteachiam.com)

Other Strategies Related to Success of TEI example, Stand for Children has launched the parent leadership training program to increase parents knowledge of issues related to teacher effectiveness and school performance. Our partnerships with other community organizations also position us for longevity in reform. For instance, the United Way of the Mid-South is working with a consortium of community organizations to develop opportunities for the District and community partners to meet and share information more regularly. In June 2011, United Way convened the first coalition meeting for not-for-profit faith-based organizations to provide information about the teacher effectiveness work, including progress to date. Stand for Children, the Memphis Urban League, and United Way shared on community advocacy and discussed how to get involved in the work. Over 60 people attended, and of those, 54 signed a pledge to support teacher effectiveness and become a member of the coalition. In like manner, Communities for Teaching Excellence, a national organization for creating advocacy on teacher effectiveness, has recently partnered with the MCS. The work of the Communities for Teaching Excellence is emerging and promising as we look to our constituents to carry the teacher effectiveness reform. In upcoming months, Communities for Teaching Excellence will be polling parents and other stakeholders to establish a baseline on beliefs about teacher effectiveness. imply that the work that happens in those areas is arbitrary. In fact, the four TEI strategies are of no consequence if we cannot build policy to sustain them; communicate accurate and appropriate messages around the work; or establish a strong community advocacy base.

Ongoing Issues & Next Steps

In many cases, we are developing policy and procedures in areas where no prior work has been done. Our challenge has been to establish new policy where there is little to no evidence of what is best practice in certain areas of reform. We are faced with the dilemma of creating policy and regulations that are speculative versus creating policy and regulations that are driven by the data experiences that derive from early implementation. Despite obvious progress, we still have a lot of work to do in terms of strategic communications for the TEI. We are optimistic about the promise of the public campaign and other opportunities for us to deliver clear messages about the purpose and intended outcomes of our efforts as well as build a consistent following for the teacher effectiveness reform.

Conclusion
Our work has been successful to date because of the strides made in the areas of policy, strategic communications, and community advocacy. The strategies described here are labeled as other strategies related to the success of TEI; however, that categorization is not meant to 65

Enablers of Implementation

Enablers of Implementation
functional initiatives. Much of the staffing success can be attributed to the decisions and recommendations that surfaced in Staffing Task Force meetings. Strategic Partnerships The partnerships that we have cultivated in the name of teacher effectiveness have proven integral to the progress of implementation. The product of our partnerships with MEA and STARS is nothing less than groundbreaking. More specifically, the significant progress made for earlier teacher recruitment and hiring timelines to improve the quality of the external candidate pool and increase flexibility for placing high-potential teachers in high-need schools was facilitated by the partnership between STARS and MEA. Our partners have a vested interest in assisting us with transforming our schools and have made their own adjustments to improve their interface with our internal offices and protocols. STARS increased its internal capacity by adding personnel to improve local management and communications. They hired a Director of Staffing to work directly with the SSZ staffing as well as persons to improve customer relations and feedback to stakeholders through immediate follow-up. Likewise, the Efficacy Institute brought on liaisons to work directly with the schools and program managers for the Envoy Project. Teacher Engagement We garnered increased traction with teachers and administrators in the field as a result of our deliberate efforts to interact with these stakeholders on an ongoing basis. Through the TEI working groups, observation rubric field tests, 66

Organizational Capacity With the exit of the Parthenon Group55 in December 2010, the development of internal organizational capacity was imperative in order to sustain the momentum they initiated during the first year. Increased capacity for implementation came about, due, in part, to more personnel hired this year. Staffing additions across the board resulted in clear work stream ownership and accountability for implementtation. The DTTE hired a Project Manager to replace the Parthenon Group, and the TEM office hired two research analysts to focus on developments of the stakeholder perceptions and teacher content knowledge components of the measure. Staff was hired to spearhead the changes to the evaluation process and corresponding teacher support strategies. Task forces were formed to increase internal collaborative efforts for key strategies. The Task Force management structure afforded these offices opportunities to work together on cross55

The Parthenon Group was a project management consultant team who supported work for the TEI from November 2009 through December 2010.

Enablers of Implementation TEI Ambassadors, and TEI Institutes56, we were able to connect teachers, assistant principals, principals, and other district-level personnel directly to teacher effectiveness work. The purpose of the TEI Ambassadors program is to develop a cadre of individuals who are wellversed in the Districts reform work and who have the capacity to share information with their colleagues. TEI Ambassadors are one way we have leveraged teacher voice to drive the reform agenda. The program was designed to convene teachers for monthly training sessions to prepare to share TEI-related information with colleagues during school faculty meetings and professional learning communities. Participation in the TEI Ambassador program was voluntary. Teachers were encouraged to volunteer to participate in the TEI Ambassador program, and principals were also encouraged to nominate teachers from their buildings to serve in this capacity. Lachell Boyd, the TEI Liaison for the Department of Teacher Talent and Effectiveness, leads the TEI Ambassador program.

56

TEI Ambassadors are teacher-representatives who are charged with sharing information about reform in their school buildings.

67

Barriers to Implementation

Barriers to Implementation
cross-analytics with separate databases in the absence of one repository of teacher, school, and district-level information. Technology such as the electronic evaluation tool and Tableau are promising tools for moving us toward a time when data quality, access, and use will be transparent, accurate, and available for driving decision-making. Organizational Capacity It has been difficult to gauge internal support for the teacher effectiveness reform in terms of system-wide collaboration. The two major divisions of the District Academic Operations, Technology, and Innovation (AOTI) and Business Operations, Logistics, and Technology (BOLT) continue to strive toward seamless and interdependent operations. Despite the increasing level of collaboration between the academic and business units on the execution of district-wide reform, there are still gaps in the transactions and follow-through that are needed to meet major milestones. For example, some departments express some resistance to the reform because it means that their work has to be done in a way that it never has been done before. Similarly, the resistance is also symptomatic of the teacher effectiveness work being labeled as an academic initiative instead of MCS new platform for educating and serving the students of Memphis. We continue to uncover areas where the duplication of efforts is a specific threat to implementation. For every new strategy that is under development to increase student achievement, there are two, three, or four existing strategies that already address the issue. We acknowledge the need to take an honest inventory of what we currently have operating in the District and 68

Systemic barriers to implementation challenged many of the outcomes of the teacher effectiveness reform agenda. Despite incremental progress in some areas described here, execution against the project milestones continues to be met with some difficulty. It is important to note that the barriers listed here are not new challenges that surfaced with the launch of the teacher effectiveness work. Rather, they represent a long history of tradition and consequences related to operational practice, organizational development, and volatile bureaucracy. Data Quality, Access, and Use By and large, one of the fundamental challenges for the ongoing teacher effectiveness reform is the limited access to and dubious quality of the data that are used to drive decision-making and continuous improvement of the reform efforts. In several cases, we were limited in our capacity to take action because elements of the data were incomplete, inaccurate, or inappropriate for addressing the most salient issues. This particular challenge emerged across strategies, especially when discussing topics of value-added data, human resource data (e.g., retirement, resignation, evaluation, and tenure), and culture and climate data. We also found it difficult to do

Barriers to Implementation identify ways to eliminate and/or consolidate efforts wherever possible. This year sparked innovation at its best in terms of reform strategies, but for these areas where prior experience is sparse (if available at all), planning and execution has been particularly challenging. For most of the strategic initiatives, including the use of video cameras, electronic evaluations, and reflective practice, there are no existing statutes, policies, or practices that have been established and taken to scale in the way that we have started to do. Our efforts to make moves in unchartered waters have shown us the best and worst of ourselves. This work has allowed us to stretch our thinking and generate creativity, yet the path not taken is difficult to pave, especially for an organization that has undergone massive change in one years time. The tension between making haste to address immediate concerns or current opportunities and managing resistance to change is ever-present. Political Context To say that the reform agenda is surrounded by a volatile political context is an understatement. Perhaps the most pervasive challenge this year was the issue of an imminent school system consolidation between Memphis City Schools and Shelby County Schools. In December 2010, the Memphis City Schools Board of Commissioners voted to surrender the charter for the school district in response to the decision of the Shelby County School Board to seek special school district status57. In March 2011, a referendum vote affirmed the Boards decision to surrender the charter and transition to a
57

consolidated school system. While the details and controversies of this issue are beyond the scope of this discussion, the looming concerns and questions that surround the future of this work warrant some attention. It is difficult to reconcile the unprecedented positive impact that Memphis City Schools TEI has had on teacher effectiveness reform at local, State, and Federal levels against the threat of being absorbed into a system that has expressed little interest in teacher effectiveness reform, at least in terms of the TEI. For months, the media unleashed an unrelenting barrage of headlines, and litigious battles pose a threat to the morale and focus of those directly and indirectly involved in transforming the city schools. The Board of Commissioners voted to extend Superintendent Cashs contract until August 2013, thereby reducing the threat of changes to the administration. In the midst of the political throws, the Bill & Melinda Gates Foundation reaffirmed commitment to our teacher effectiveness agenda and the children of Memphis City Schools. During our April 2011 Stock Take with the Foundation, we were lauded on our ability to reach milestones despite the political distractions that could have easily thrown our work off course. Other changes in the Legislature made this a year of uncertainty and anxiety around aspects of the reform agenda. Specifically, changes to the tenure laws have overshadowed the tone of support and continuous professional growth that is embedded in our reform efforts. All of these issues could potentially be a huge distraction to internal and external stakeholders who have a vested interest in the reform work. As we keep focus on the work, our priority is to keep our stakeholders attention on the reform and maintain support for the school system and the children of Memphis. 69

In 2010, the Shelby County Schools Board of Education sought legislation to become a special school district. The legislation would enable the Board to freeze boundaries and levy property taxes for schools.

References

Bill & Melinda Gates Foundation. (2010). Learning about teaching: Preliminary findings from the Measures of Effective Teaching Project. Available at http://metproject.org/downloads/Preliminary_Findings-Research_Paper.pdf. Bill & Melinda Gates Foundation. (2010). Working with teachers to develop fair and reliable measures of effective teaching. Available at www.gatesfoundation.org/highschools/Documents/met-framingpaper.pdf. Hamer, I. (2011, April 28). Memphis schools make quick gains [Letter to the editor]. The Tennessean. Retrieved May 3, 2011, from www.tennessean.com. Memphis City Schools. Measures of Effective Teaching (MET) Preliminary Results. From 30K: A Weekly Briefing from Research, Evaluation, Assessment, and Student Information (REASI), 3 (14). Available at http://www.mcsk12.net/aboutmcs_30K.asp. Walker, Kristin M. (2010). Case Study 2010: Teacher Effectiveness Initiative. Available at www.mcsk12.net/tei.

70

Resources

Battelle for Kids www.battelleforkids.org Center for Transformative Teacher Training www.transformativeteachertraining.com Communities for Teaching Excellence www.pbis.org www.4teachingexcellence.org RANDA Solutions Efficacy Institute http://randasolutions.com www.efficacy.org Stand for Children I Teach. I Am www.stand.org www.iteachiam.com TEI Advisory Board Measures of Effective Teaching (MET) Project www.mcsk12.net/tei/board.asp http://metproject.org Memphis City Schools Mediasite http://mediasite.mcsk12.net TELL Tennessee Conducting More Rigorous Teacher Evaluations and Navigating the Evaluation Process is available at http://mediasite.mcsk12.net/mediasite 5/Catalog/pages/catalog.aspx?catalogId =f06e4d93-f18e-49ac-bbe81192588b28b5 Memphis City Schools Policies www.mcsk12.net/policy/policy.asp www.telltennessee.com The New Teacher Project www.tntp.org ThinkShow! www.thinkshow.org Tripod Survey www.tripodproject.org/ Tennessee Department of Education http://state.tn.us/ New Leaders for New Schools (NLNS) www.nlns.org Positive Behavioral Intervention and Support (PBIS)

71

Acknowledgments
There are so many colleagues, community partners, expert practitioners and stakeholders who have been instrumental in the development of this case study. I will not endeavor to list everyone here, but their specific and unique ties to the story of Memphis City Schools reform are noteworthy. They enrich our story as thought leaders and critical friends. I want to also acknowledge the continued advocacy for the knowledge capture and sharing with regard to teacher effectiveness reform from the MCS senior management team, specifically Superintendent Cash and Deputy Superintendent Hamer who believe in our story. Lastly, I want to extend many thanks to Dr. John Amis, Dr. Celia Anderson, and Dr. Beverly Cross, panel of research advisors who continue to support and guide my work in case study development

72

Department of Teacher Talent and Effectiveness Directory

Phone: (901) 416-0135 Email: tei@mcsk12.net

Tequilla Banks Executive Director Lachell Boyd TEI Liaison Lee Brother Special Projects Coordinator I TEI External Partners Jennifer Chandler Special Projects Coordinator II Reflective Practice Marqui Fifer Special Projects Coordinator II Teacher Evaluations and Tenure Sherrish Holloman Coordinator Teacher Support, Retention, and Recognition Carla Holloway Coordinator Teacher Evaluation and Tenure Donna James Executive Assistant to Tequilla Banks Monica Jordan Coordinator MET/Reflective Practice 73
58

Jessica Lotz Special Projects Coordinator Mike Neal Coordinator Career Management Miesha Turner Executive Assistant Internal Programming Kristin Walker TEI Archivist

Office of Teacher Effectiveness Measure (TEM)58

Rorie Harris Coordinator of Teacher Effectiveness Measurement Tracy Brittmon Research Analyst Tracey Wilson Research Analyst

The TEM Office reports to the Department of Research, Evaluation, Assessment and Student Information (REASI).