District evaluates progress in the takeover schools
Philadelphia’s reform plan that overhauled the management of 70 schools has been described as an historic experiment. Amid public outcry, private companies and nonprofits were brought in to manage some schools, while others were converted to charter schools or restructured under District management.
Now, as Year One of the reform draws to a close, many wonder whether the sweeping changes have translated into better student performance, and how the School District is measuring improvement.
The main response from District officials so far is that it is too early to say anything about student performance.
A closer look reveals that a varied and confusing array of standards and processes has been created to monitor the experiment and hold schools accountable. While evaluation within the District is taking place, little information has gotten to the public to date, and key structures identified to engage the public have not been fully implemented.
SRC laid out standards
To ensure that the "partnership schools" targeted for a change in management actually improve educational opportunities, the School Reform Commission (SRC) outlined standards in a detailed, six-page resolution adopted last April, when the new management structure was approved.
The standards cover many areas including: school improvement plans, community involvement, teacher qualifications, class size, student mobility, test scores, and access to information.
According to the resolution, all educational providers have to submit school improvement plans and meet their stated yearly objectives — including provisions for more qualified teachers, smaller class sizes, and improved student-teacher ratios — or face termination of their contract.
Further, the District stipulated that 50 percent of partnership school students would have to meet or exceed the state average on the PSSA standardized exam by 2005, and that students be provided a transfer option if their needs cannot be met by their assigned schools.
The resolution specified that all information regarding partnership schools and student progress not considered confidential be treated as public information and available to all.
Internal process in place
District officials stated that they are utilizing the April 2002 resolution’s standards as much as possible to evaluate provider contracts.
Deputy Chief Academic Officer Ellen Savitz indicated that evaluations could not be made on test score criteria, because spring test results are not in, or on the "certified teacher in every classroom" criterion because there was a districtwide teacher shortage. However, she said District staff and the SRC did use the resolution’s standards on reduced class size as part of their evaluation of school managers’ performance. They also examined whether terms of the contracts with educational management organizations were being met.
Based on the evaluation performed by Savitz and her Office of School Development, District CEO Paul Vallas and the SRC fired one manager, Chancellor Beacon, and rewarded three school managers, Victory Schools, Temple University, and Foundations Inc., with additional schools.
"The reason we created an Office of School Development is to monitor very closely the charter schools and EMOs and to really look at the actual operations of the schools and whether the schools are living up to their contractual obligations," Vallas stated.
The District’s key method for measuring schools’ academic progress is a program of school visits supported by SchoolWorks, a Massachusetts-based education consulting company.
The quality review process calls for a team of District central and regional office staff to visit every partnership school. The school visits are designed to "determine a baseline and then measure progress in schools across the District," according to Chief Accountability Officer Joseph Jacovino.
Evaluators attended two training sessions to learn the detailed evaluation procedure, which includes working in teams to review school data, observe classes, and interview school personnel. After each school visit, the team leader writes a summary report based on the findings of the team and meets with the school leadership to discuss findings. The school must then create a corrective action plan, and the District conducts follow-up visits and conversations to assist with and assess implementation.
The teams are slated to complete close to 100 visits by the end of the year, including visits to a group of charter schools and high schools, and to the 16 schools receiving extra funding. Five higher performing schools are being visited to provide comparison data.
Public left in dark
While the SchoolWorks visits have taken hold in the District, key structures for informing and engaging the public in evaluating the reform effort are still lacking.
The April 2002 SRC resolution stipulated that information regarding student and school performance would be public information and available upon request. Partnership schools were also required to report twice a year on trends in suspensions, expulsions, and involuntary transfers to ensure that student improvement was not manufactured by changing the student population.
However, this mandate has been stymied thus far by inaccurate data, red tape, and altered policies.
The District admitted to the Daily News in early April that its attendance and disciplinary data are not valid due to errors in collection and recording. Vallas stated, "I don’t think the data is reliable, it’s not accurate."
Regardless of accuracy, community groups say information is hard to come by. One organization, Eastern Pennsylvania Organizing Project (EPOP), initiated a "Right to Know" campaign soon after the takeover took place, pressuring the District to improve its communication with parents.
"Getting information from the District still requires going through hoops," said Delores Shaw, a parent leader with EPOP. Shaw said communication with parents and access to information is "very inconsistent from school to school."
Additionally, District officials say the mandated twice-a-year reports on student discipline and movement have been "superceded" by Vallas’s mandatory reporting policy, which requires schools to report all instances of serious disciplinary action. But the SRC resolution’s requirement that data on student mobility be publicly reported, aimed to help identify trends of discriminatory admissions, has not been addressed.
Asked about this change in reporting, SRC member Sandra Dungee Glenn stated, "The Accountability Review Council will be creating reports based on their evaluation that may note those kinds of trends."
But so far there is no Accountability Review Council.
Creation of the Accountability Review Council was mandated by SRC resolution in December 2002, at the urging of State Senator Allyson Schwartz of Northwest Philadelphia. The Council will measure how schools are meeting the improvement standards, and publish its findings annually in reader-friendly reports. It will consist of seven members plus two consultants, Jacovino said.
According to Schwartz, the Council is designed to "give the public, and in particular parents of partnership school students, an accountability mechanism, independent from the School District."
But District officials are still reviewing candidates in a process the Senator’s office has described as "lengthy." Schwartz said she has been assured that the Council would be appointed by the end of the school year.
The Council is not set to issue its first reports until December 2003.
Community partners cut out
Pairing partnership schools with community-based organizations was another strategy outlined in the SRC’s April resolution for the community to monitor and have input in the reform.
The partnerships never materialized, though the District did receive proposals from organizations across the city. According to Maia Cucciara, researcher with the nonprofit organization Research for Action, not only were community partners not assigned in any takeover schools, but the SRC never responded to interested groups’ applications.
Thus, it has been left to each individual school and school provider to create and sustain parental and community involvement as they choose, and there are no funds to facilitate partnerships.
The ultimate accountability lever offered to parents by the SRC’s April resolution was the opportunity to vote with their feet if they were unhappy with a school.
Partnership school students and parents were mailed letters last spring offering a "one-time only transfer option" at the end of the school year, according to LeTretta Jones in the District’s student placement office.
Jones said so few transfer applications were submitted that the office was largely successful in placing those students. She acknowledged there was a short turnaround between the time parents were informed of the option and the deadline for requests.
This spring, students at any of the District’s 176 lowest-scoring schools, as designated by the state, could apply for openings in 20 higher-performing schools.
Asking the right questions?
The impact of the accountability measures that have been implemented remains to be seen.
According to Commissioner Glenn, the SRC will have to get more information to know if the procedures in place are sufficient.
"Until we compile a complete set of data we will not be able to get a full picture," she commented. "Right now, we are not even sure if we are asking the right questions."