MONTH YEAR

Evaluation of the Common Core

Technology Project
Interim Report
Executive Summary
Jonathan Margolin, Ph.D.
Erin Haynes, Ph.D.
Jessica Heppen, Ph.D.
Kristin Ruedel, Ph.D.
John Meakin
Alison Hauser
Jarah Blum
Suzette Chavez


September 2014





Evaluation of the Common Core
Technology Project: Interim Report

Executive Summary

September 2014


Jonathan Margolin, Ph.D.
Erin Haynes, Ph.D.
Jessica Heppen, Ph.D.
Kristin Ruedel, Ph.D.
John Meakin
Alison Hauser
Jarah Blum
Suzette Chavez











1000 Thomas Jefferson Street NW
Washington, DC 20007-3835
202.403.5000 | TTY 877.334.3499
www.air.org

Copyright 2014 American Institutes for Research. All rights reserved.
3121_09/2014

American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP1
Executive Summary
The Common Core Technology Project (CCTP) is the Los Angeles Unified School District
(LAUSD)s signature investment in technology, with plans to deliver technology devices to
every teacher and student in the district. The CCTP aims to transform learning throughout the
district by providing interactive and engaging learning environments, supporting implementation
of the Common Core State Standards with digital curriculum materials and assessment, and
closing the Digital Divide by ensuring that all students have access to 21st century technology.
American Institutes for Research (AIR) is conducting an external evaluation of the project to
address the implementation and outcomes of the program. This Interim Report is intended to
provide formative feedback to inform program improvement based on the evaluation of the
programs first year of implementation. It addresses the period from August 2013 through June
2014. In August 2013, the district deployed iPads to an initial wave of 47 Phase 1 schools. The
devices were preloaded with curriculum content designed by Pearson Education to be aligned
with the Common Core State Standards. External vendors (Pearson and Apple) and LAUSD staff
provided two- to three-day professional development sessions in summer 2013, as well as in-
school training for teachers throughout the year. LAUSD hired 14 Virtual Learning Complex
Facilitators (VLCFs) to prepare leadership teams at each school to support the integration of
Common Core Technology Resources in alignment with Common Core State Standards. The
district also assigned 14 Microcomputer Support Assistants (MCSAs) to provide technical
support focusing on configuration of devices, wireless connectivity, and hardware or software
malfunctions.
The evaluation questions addressed in the interim report are as follows:
1. How is the technology being used by teachers and students in CCTP schools and other
school-based technology-integration initiatives?
2. What is the nature of the districts planning and support of CCTP and other school-based
technology-integration initiatives?
3. What are the early activities, experiences, and perceptions of principals and district staff
regarding the technology applications?
4. Based on a synthesis of the findings, what are the recommendations to the district?
Data for this report were collected in spring 2014. Data sources included the following:
Key district documents regarding CCTP planning and implementation
Interviews with 12 district leaders including leaders of the CCTP team, and six of the 12
VLCFs
Focus groups with five MCSAs and five Educational Service Center (ESC) area
superintendents
Classroom observations in a mix of grade levels and subject areas, in a sample of schools
that included 15 Phase 1 CCTP schools and four non-CCTP schools implementing other
technology initiatives
American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP2
Mobile Device Management data for all CCTP Phase 1 schools to capture device usage
by teachers and students
Our findings are summarized in four sections: Technology Use in Schools, District Leadership,
Technical Support for Schools, and Instructional Support for Schools. The findings presented in
this summary reflect points of convergence of multiple respondents or data sources; we mention
the particular data source where this information is particularly relevant for interpreting the
finding. We conclude with a discussion of implications for the further rollout of CCTP.
Technology Use in Schools
In this section, we summarize data collected on technology use in visited schools. Because these
findings are intended to serve as a baseline for the evaluation, we do not propose
recommendations at this time.
Level of Technology Use. In the 15 CCTP schools we visited in May 2014 to interview school
staff about technology use and to conduct classroom observations, iPads were present in 79
percent of classrooms and in use in 48 percent of classrooms. In the four non-CCTP schools,
laptopsthe most prevalent form of technologywere observed to be present in 60 percent of
classrooms and in use in 28 percent of classrooms. The most frequent use of iPads and other
devices in the observed classes were: technology-enabled whole-class instruction delivery, the
practice of mathematics or English language arts (ELA) skills via game-like programs (e.g., ST
Math and Lexia Reading Core5) and other apps, the creation of products or projects, and general
use as a learning resource (e.g., calculator, word processing). With regard to the use of specific
apps on the iPads, students were most frequently observed using tools such as the calculator,
Notability, iMovie, and Keynote. Apps related to ELA and mathematics skills practice were the
next most frequent. Schools were not implementing the Pearson curriculum because of delays in
access to the curriculum and technology problems with logging in.
Examining the districts device usage records, we found varying levels of student iPad usage in
Phase 1 CCTP schools, where usage was defined as the proportion of devices per school connected
to the network during one week in June 2014. Usage data showed that use was higher in
elementary than in secondary schools, a finding that may be the result of several probable causes.
For instance, some high schools had put away devices for the year in spring 2014, and some did
not have or use iPads much if at all during the 201314 school year.
Across Phase 1 CCTP schools, teachers and students frequently downloaded apps that serve both
academic and nonacademic purposes. Among the academic purposes, the most frequently
downloaded apps were lesson platforms (and the related purpose of document sharing) and
programs that provide mathematics or ELA content.
Promising Practices for Using Technology. School staff noted numerous promising practices
related to student and teacher uses of technology. Promising student practices include the following:
Project-based learning: movies, animations, story writing, drawing
E-mail communication with teacher and submission of homework assignments
Adaptive learning programs
American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP3
Virtual field trips: Students use the Internet to connect with people and places around the
world
Promising teacher practices include the following:
Interactive lesson content (e.g., movies, problem sets) arranged by teachers and pushed to
student devices, combined with formative assessments to check for understanding in
real time
Communication with students at any time
Submission of student papers, grading, and immediate feedback
In-class chat function for student questions
Recorded lessons available for viewing by students who were absent and for review
Use of Technology to Support Assessments. Overall, schools experienced common early
implementation issues, such as learning about device readiness and technical glitches with the
Smarter Balanced Assessment Consortium (SBAC) field test. These initial hurdles included
problems navigating the interface, login, and scheduling conflicts with instructional time.
District Leadership
The CCTP is a large and complex initiative. The project is housed in the Office of Curriculum,
Instruction, and School Support (OCISS), and the project team comprises more than 50
individuals distributed among the following six functional groups: deployment,
instructional/content development, safety, organizational change management, technical, and
project management. The project team coordinates with several other departments, such as the
Information Technology Division (ITD), Facilities Services Division, and the Los Angeles
School Police Department (LASPD). In this section, we provide a summary of district leadership
pertaining to four topics: deploying devices to schools, maintaining safety and security of
students and devices, coordinating with related initiatives, and communicating about the project.

Deploying Devices to Schools. Deployment of devices to Phase 1 schools was a central goal of
the initiative in the first year. A deployment team, consisting of two leads assisted by VLCFs and
MCSAs, oversaw the deployment of iPads to 30,490 students and 1,360 teachers in 47 Phase 1
schools. The challenges of deployment on this scale meant that project staff not on the
deployment team needed to spend their time on technical troubleshooting rather than supporting
technology integration into instruction in the first year of implementation. A key challenge was
the time required to set up devices for individual users. Other difficulties arose from lack of
technological readiness of schools (staff and infrastructure). Some of the data collected in spring
2014 suggest that the current deployment model is not sufficient to meet the goal of full-scale
deployment throughout the entire district on the currently envisioned schedule. The evaluation
team recommends that, to make deployment run more smoothly, the district should find a
technical solution to decrease the time spent on provisioning each device. Another
recommendation offered below about increasing support staff also is relevant to deployment, as
there were many technical support requests during the deployment period (see Technical
Support for Schools).

American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP4
Maintaining Safety and Security of Students and Devices. The district had a four-pronged
strategy for safety and security, focusing on cyber safety, community outreach, technology
solutions, and collaboration with law enforcement agencies. Regarding cyber safety, CCTPs
Digital Citizenship campaign aimed at students, parents, and school staff sought to promote
awareness and education about navigating an online environment safely and responsibly. There
were several aspects of this campaign, including presentations to principals; a section of the
CCTP website with Digital Citizenship resources; parent trainings; and Digital Citizenship
Week, a weeklong series of special events held in April 2014. The district engaged in community
outreach about safety through meetings with parents and students and through public service
announcements on public television. The district used technology to deter theft; each device
could be remotely tracked and disabled in case of loss or theft. LASPD coordinated the Law
Enforcement Working Group, a collaboration among local law enforcement agencies,
prosecutors, and community organizations, to address issues such as device theft, summer
storage, and student safety.

In September 2013, students at three high schools disabled content filters in CCTP-provided
iPads. In response to this incident, the district required that devices stay at school. This move
created logistical challenges for schools to distribute and collect devices on a regular basis. This
challenge was particularly pronounced in secondary schools, where students change classrooms
and teachers throughout the day. The decision to keep devices on school grounds also made the
school the custodian of the devices on nights and weekends. Some district leaders stated that this
restriction may be removed for the 201415 school year. In the meantime, middle and high
schools could benefit from learning about other schools trials and errors in handling the
challenges created by this restriction. The district should consider establishing a secondary-
level task force to address logistical challenges of iPad distribution and monitoring.
Representatives from schools that have struggled in this area, and/or have developed increasingly
effective procedures and protocols for distributing and tracking iPads, could serve on a task force
that advises the district-based CCTP team and disseminates information to other CCTP middle
and high schools.

In addition to concerns about device tracking and security, several school staff expressed concerns
about student safety and device security. These concerns related to the possibility of theft from
students (if they are allowed to take iPads home) and from schools. School staff also expressed
concerns for students personal safety related to access of unsanctioned content on the Internet
when unsupervised. Allowing students to take devices home can be an important step in giving
all students equal access to technology resources. However, the district should leverage the
insights of administrators, teachers, and parents at CCTP schools before making the
decision to allow students to take the devices home to ensure these concerns are addressed.

Coordinating With Related Initiatives. The CCTP director and other district-level instructional
directors (e.g., curriculum and instruction, special education) are responsible for coordinating
with ongoing instructional initiatives through CCTPs Instruction Committee. There is some
evidence that coordination has been minimal or nonexistent with respect to professional
development, either with the instructional directors or ESC area superintendents for instruction.
One reason for this is that professional development offered during 201314 related to the
Common Core was meant to be applicable to teachers in all schools in their regions of the
American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP5
district; it was not targeted specifically at integrating technology resources provided by CCTP,
which would have been applicable only to the Phase 1 schools. Perhaps as a result of the lack of
coordination, school staff did not perceive how CCTP supports implementation of the Common
Core. The evaluation team recommends that the district integrate training about the Common
Core standards into training about technology and vice versa. The district CCTP team,
perhaps with representatives from CCTP pilot schools, should collaborate with district Common
Core leaders and trainers to make sure technology is presented as a key tool in Common Core
implementation and, conversely, explicitly include a focus on the Common Core standards and
their implementation in all CCTP trainings. For example, professional development modules
being developed by VLCFs could incorporate Common Corerelated information and resources.

Communicating About the Project. District staff indicated that a key early implementation
goal was to communicate effectively about the initiative with schools and with the broader
community. The district has made progress toward this goal but has encountered challenges with
both audiences. Communication with schools consisted of a monthly meeting of CCTP leaders
and school administrators, phone conferences and e-mails with school administrators as needed,
and a monthly CCTP newsletter. VLCFs have been the primary point of contact with Phase 1
schools on technical issues. Staff at several schools noted communication problems regarding the
initiative, such as lack of guidance about the purpose of the devices along with apps they should
use and how, when the devices would be deployed, and whether students would be able to take
them home.

As for reaching a broader public, several aspects of the project have been the subject of public
scrutiny, such as the use of public bond funds, selection of a single vendor and device platform,
and breaches in device security and tracking. Media attention on the project required reactive
attempts by the district to respond. To improve and coordinate CCTP-related communications
with the public, the LAUSD Joint CCTP Communications Task Force was established in fall
2013. The Task Force created several communication vehicles to explain the initiative, including
a project website (http://achieve.lausd.net/cctp), monthly newsletter, and several hourlong
television programs about the project, produced by LAUSD and aired on public television station
KLCS. At the end of the 201314 school year, several district leaders acknowledged that
communication with the broader public remains a challenge for CCTP. In later phases of the
evaluation, we will further examine the roles and responsibilities of district leaders with respect
to communication with schools, parents, and the broader public.
Technical Support for Schools
One of the largest challenges for early implementation of CCTPas with other state and district
technology initiativeshas been technical issues, including device access and wireless
connectivity. The district provided support to Phase 1 CCTP schools through VLCFs, MCSAs,
and the ITD Help Desk. Both VLCFs and MCSAs indicated their respective responsibilities were
overlapping in some cases. By winter 2014, school staff were able to submit help requests
directly to the Help Desk, although initially only the VLCFs or MCSAs were able to submit
these requests. The most frequent types of Help Desk requests were requests for new iPads (e.g.,
for new students), requests to reset disabled iPads (e.g., student forgot password), and requests to
fix a problem with device management software (typically when a device was not registered in
American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP6
the system). Nine of the 15 Phase 1 CCTP schools we visited had designated a staff member to
serve as a point of contact for technical problems.

Access to technical support was a concern at several schools that we visited as part of the Year 1
evaluation activities. Staff at six of the 15 Phase 1 CCTP schools reported general difficulty
reaching district personnel assigned to their school, as well as difficulty obtaining follow-up from
district personnel to resolve technical issues. The CCTP schools we visited reported having different
processes for accessing support, some of which involved immediate communication (e.g., e-mails
and phone calls to VLCFs) or delayed communication (e.g., dropping off malfunctioning devices in
the main office). The Help Desk, although within industry standards of approximately a one-week
turnaround time, is not adequately meeting schools needs. A majority of VLCFs stated that
additional technical support staff are needed to adequately address the technical support needs of
participating schools. In terms of underlying reasons for the need for technical support, VLCFs
perceived that some schools lacked technology readiness with respect to infrastructure and staff
proficiency with technology. To address the need for more prompt and effective technical
support, the district should ensure that there are a sufficient number of MCSAs assigned to
the project, and clarify the process by which schools access technical support. The district
also may consider assisting CCTP schools in building a trained team (or individual) to
collaborate with the MCSAs and VLCFs to provide technical support, including
troubleshooting issues, communicating with the ITD Help Desk, and providing technical support
directly to school staff as needed. There is evidence that the district is taking steps to ensure that
devices are rolled out to schools that meet certain technological readiness criteria (with
corresponding efforts to build technological readiness in remaining schools). This effort seems to
be a promising approach to reducing the volume of requests for technical support.
Instructional Support for Schools
Teachers received support for integrating technology into instruction from different sources in
the first year of CCTP implementation. First, the district provided a mandatory two- to three-day
training session for all certificated staff in CCTP Phase 1 schools in August and September 2013.
This session focused on device setup, use, and management, and introduced the Pearson digital
curriculum content. During the school year, Pearson also provided on-site coaching and technical
assistance on integrating Pearson digital content into curriculum and instruction. However,
school staff reported that Pearson curriculum materials were not complete at the time of training
or that the training did not adequately prepare them for integrating technology into their
instruction. In general, our findings from the first year suggest the district should provide
teachers with a variety of professional development approaches about how to successfully
integrate technology into classroom instruction. Teachers and other school staff do require
training on accessing and using the devices and apps, but CCTP professional development
should evolve to address more educationally substantive aspects of implementation.
The source of technology-related professional development most frequently reported by school
administrators and school technology coordinators was peer-to-peer training, often through
school leadership teams. Some school staff indicated that this was the most useful professional
development provided to teachers about integrating technology into the classroom. The district
expected the VLCFs to prepare school leadership teams both prior to and following deployment,
but VLCFs have, in most cases, not yet worked with these leadership teams. We recommend
American Institutes for Research Interim Report Executive Summary: Evaluation of the CCTP7
that the district encourage the formation and active involvement of school leadership teams
and professional learning communities. For example, VLCFs could help encourage teachers to
visit each others classrooms and schools, and work collaboratively during common planning
time and regularly scheduled meetings.
Instructional support for technology integration is essential for achieving the goals of CCTP. The
preponderance of evidence suggests that the challenges of deployment and technical support
drew the focus of VLCFs, as well as district CCTP leaders, away from supporting instructional
practice to focusing on troubleshooting technical problems. Therefore, the district should create
opportunities for VLCFs and district leaders to support technology integration into
instructional practice. There are a number of possible approaches to enhancing the focus on
instruction. If resources allow, hiring additional MCSAs would reduce the immediate burden of
technical support. Ensuring that schools are technologically ready before deploying devices
might reduce the volume of requests for technical support. Finally, the CCTP leadership team
should clarify the appropriate roles and responsibilities for the deployment and
instruction/content developer leads, to ensure they are devoting their efforts to instructional
support in the manner and to the extent originally envisioned.
Discussion
The challenges discussed above should be addressed promptly, as they impede progress in Phase
1 schools and are likely to impede implementation and subsequent progress in the next phases.
Of greatest importance is making sure that adequate support structures are in place, both at
school and district levels. Adequate support occurs at three levels. First, it is crucial to ensure
that schools are technologically ready (i.e., that adequate infrastructure is in place) before
deploying devices. Second, school- and district-level support systems and individuals should be
prepared to handle technological issues that arise (e.g., nonworking devices) and should provide
training to teachers and other staff who are not familiar with using the devices. Third, to move
toward the goals of transforming instruction with technology and ensuring that students have
access to 21st century technology and innovative applications of technology to teaching and
learning, it is not sufficient to simply provide classroom access to devices. Teachers must be
prepared to integrate the technology into instruction, redefining how they present and design
instruction. School- and district-level training and professional support must therefore be geared
toward developing teachers skills in this area. Finally, as the district brings CCTP to full scale, it
will be important to have a fully developed communication and feedback plan in place to ensure
that these support structures are happening as planned in every school, and are perceived to be
effective by various stakeholders.