2007-2008 AISD REACH Evaluation Objectives

Evaluation objectives for pilot Year 1 include the following:

• To document the accomplishment of Year 1 Operational Goals.
• To provide formative evaluation for each program element.
• To establish baseline data for pilot and comparison schools.
• To define the scope of research for Year 2 through 4.

Scope and Method

Data Collection

Information will be collected throughout the school year from teachers, principals, and students. Teachers will be surveyed during the Fall semester regarding their opinions about compensation, attitudes toward teaching, mentoring, and professional development. All staff will be surveyed about the campus climate late in the Fall semester, parents will be surveyed early in the Spring, and all students in grades 3-11 will be surveyed about campus climate late in the Spring semester. A sample of principals and teachers will participate in focus groups regarding the pilot initiative. All SLOs from each campus will be rated for both quality and rigor, to establish baseline information.

Data Analyses

Data analysis procedures will include summaries of survey responses and ratings of SLO quality and rigor. Focus group data and survey data will be examined for themes and summarized for formative evaluation purposes. Correlations and other appropriate analyses will be performed to assess the relationship between SLO quality and rigor, TAKS, and other relevant indicators.

The results of the Districtwide surveys will be examined to monitor parent and staff attitudes toward compensation. In addition, the AISD Teacher Survey will be analyzed to compare teacher attitudes toward teaching, compensation, mentoring and other key variables between SCI Pilot schools and other AISD schools.

Time Line

• November 2007: Teacher Survey is conducted; comparison schools are selected.
• November 2007: Staff Climate Survey is conducted.
• December 2007: Principal focus group is conducted.
• January/February 2008: Teacher focus groups are conducted.
• February 2008: Parent Survey is conducted.
• March/April 2008: Student Climate Survey is conducted.
• April 2008: Employee Coordinated Survey is conducted.
• July 2008: Year 1 Scorecard is completed; Formative Evaluation Summary is completed.

Reporting

A score card will be completed to report the accomplishment of Year 1 Objectives and a narrative formative evaluation summary will be prepared to identify successes and recommendations for improvement based on the first year. In addition, several interim reports will be issued to summarize the results of surveys and focus groups.

Program Support

Program Evaluation staff will assist in the development of tools such as rubrics for Student Learning Objectives, training materials for staff, and an Innovation Configuration tool to be used for Mentor performance evaluation. Program Evaluation staff will assist in the preparation of an RFP for a new formative assessment that will measure student growth, and will provide information as needed to program staff and pilot participants. Finally, Program Evaluation Staff will coordinate with researchers conducting the external evaluation.

External Evaluation

In cooperation with the Department of Program Evaluation, researchers at the National Center on Performance Incentives (NCPI) will conduct an external evaluation of the Strategic Compensation Pilot program.

(from the NCPI website)

Overview

Established in 2006 through a $10 million, five-year research and development grant from the United States Department of Education's Institute of Education Sciences, NCPI conducts independtent and scientific studies on the individual and institutional effects of performance incentives in education. NCPI, a state and local policy research and development center, is part of the Peabody Center for Education Policy and housed in the Learning Sciences Institute on the campus of Vanderbilt University's Peabody College, one of the nation's top five schools of education.

NCPI joins the National Research and Development Center on School Choice at Peabody College, making Peabody the only institution in the nation to house two national research and development centers funded by the Institute of Education Sciences.