A distorted reality
An ex-principal says inflated test scores skewed decision-making and hurt students. The problem isn’t fixed.
By by Benjamin Herold for NewsWorks, a Notebook news partner on Nov 20, 2012 05:40 PM
In July 2010, when Saliyah Cruz was named principal of Communications Technology High, state test scores said the small citywide admission school in Southwest Philadelphia was one of the best in the city.
Everything else said something different.
SAT scores were poor. Summer enrichment programs were empty. Loads of kids tested into remedial reading and math. According to Cruz, even the police complained that many students had spent much of the previous school year at the nearby Penrose Plaza strip mall instead of in class.
“Those kinds of things didn’t add up for me,” she said. “If my kids were out in the street when they belong in school, how were they scoring  percent proficient?”
Two years later, an answer appeared: A mountain of circumstantial evidence now suggests that Comm Tech’s results on the 2010 Pennsylvania System of School Assessment (PSSA) exams were inflated by adult cheating. The school is one of 53 District schools and four area charters involved in a state-led investigation that has prompted questions about the validity of test results between 2009 and 2011.
Cruz, now a middle school principal in Delaware, says the suspect scores at Comm Tech hurt students. She also believes they reflected a districtwide culture of rewarding improbable PSSA gains while dismissing steady improvement.
“The message quite clearly was, ‘Here’s what’s expected in the School District of Philadelphia,’” said Cruz. “All the principals, all the teachers, all the kids need to be able to make these giant leaps forward.”
Testing experts say the ripple effects of inflated scores go even wider, especially because the District continues to rely heavily on data that is likely tainted to measure success and make high-stakes policy decisions.
“I think the implications are pretty profound,” said Jonathan Supovitz, a University of Pennsylvania professor who co-directs the Consortium for Policy Research in Education (CPRE).
“If we can’t assume the stability of the data, then any sense of guidance about what we’re doing well or not well is broken down.”
District officials declined to be interviewed for this story.
“It is too early to say how the PSSA scores have been affected by the allegations of testing improprieties,” wrote spokesman Fernando Gallard in a statement, citing ongoing investigations.
Each year, students in grades 3-8 and 11 take the PSSA in reading and math. Their scores are used to determine whether schools meet federally mandated performance targets, known as adequate yearly progress (AYP). In Philadelphia, they’re also used to make big decisions, including which schools get closed or converted to charters.
In 2010, 75 percent of 11th graders at Comm Tech scored proficient or above in reading. That was a 22 percentage-point jump over the previous year.
In math, 70 percent of Comm Tech 11th graders scored proficient or above, 40 points higher than the year before.
An analysis commissioned by the Pennsylvania Department of Education suggested the results may be illegitimate. In both 2009 and 2010, a high number of student response sheets at Comm Tech had suspicious patterns of “wrong-to-right” erasures – a telltale sign of adult cheating.
Before the 2010-11 school year started, Comm Tech’s principal, Barbara McCreery, was replaced. That year, under Saliyah Cruz, the suspicious erasures went away. The school’s scores tanked, dropping 38 points in reading and 45 points in math.
McCreery, now the principal at Bok Technical High, declined to comment for this story.
Cruz says she wasn’t sure what to think when she walked into Comm Tech.
“I thought I was taking the helm of a high-performing school,” said Cruz. “Although there were some red flags.”
She says she tried talking to Comm Tech staff to get a handle on what was going on.
“‘Guys, help me understand this. What were we doing last year that accounted for the kind of academic performance the kids had?’”
In response, says Cruz, staff pointed to “Study Island,” a computer-based test prep program used at many District schools.
“It didn’t make any sense,” she said.
Despite her skepticism, Cruz says the 2010 PSSA results still led her to believe that only a small proportion of Comm Tech’s students needed remedial help. Rather than overhaul staffing patterns and course schedules to allow for a schoolwide intervention, she expanded use of Study Island.
But early indicators signaled disaster. Reports generated by Study Island suggested that students didn’t understand the material. Interim tests used to predict PSSA performance pointed to huge score drops. Cruz’s own eyes told her that students weren’t learning.
Some of her staff refused to believe any of it, she says.
“I got a lot of pushback,” said Cruz. “‘I don’t care what all this data is saying, our PSSA scores say something different.’”
Her efforts to get some staff to change their instruction or re-teach content were rebuffed.
“I felt like I was running into a brick wall,” said Cruz.