From AERA: Are the schools in Philadelphia trending in the right direction?
The American Educational Research Association conference is April 3-7 at the Pennsylvania Convention Center and the downtown Philadelphia Marriott hotel. This excerpt is from a presentation on Friday by Notebook editor and publisher Paul Socolar in a session about "The Landscape of Education Reform in Philadelphia." The topic was whether there are positive trends in school performance in Philadelphia.
Yes, since the state takeover in 2002, the trends are positive on a number of indicators … and not just test scores. Graduation rates are up – now, finally, two-thirds of students are graduating high school within six years. And more of the graduates are going to college.
But the story of how Philadelphia schools are doing is complicated and much murkier.
My vantage point for this is as editor for the past 15 years of the Notebook. We’re an education newspaper whose mission is to help grassroots communities understand what’s going on in this troubled school system -- so they can become active and engaged in it. The Notebook pays close attention to data and tries to figure out what’s useful and not misleading.
As Philadelphia has been on the cutting edge of reform, it’s also on the cutting edge of high-stakes accountability. Schools have been under intense pressure to produce good results -- to produce good numbers. Student achievement data are often published for the purpose of battles over the direction of school reform. So the Notebook has been witness to more than a few efforts to manipulate or selectively use the data.
I attended all the annual School District press conferences where for nine straight years, from 2003 to 2011, they announced rising test scores in reading and math with total gains of 20-40 percentage points during that span. There was no explanation why it was that 3rd-grade reading scores on the state test, the PSSA, were up 20 points, but the scores on the DRA, the Developmental Reading Assessment, administered by teachers to those same 3rd graders every year, were essentially flat.
We heard from school staff that the pressure they were under to raise scores and some ways that teachers and schools taught to the test and gamed the system. That was why in 2011 we investigated whether the state had ever done a forensic analysis of the answer sheets on the state test.
We discovered that there had been such a study in 2009 that had been essentially buried, and when we got our hands on it, it showed that statistically improbable patterns of wrong-to-right erasures were rampant in schools across the state. That was the beginning of Pennsylvania’s cheating scandal; dozens of districts were ordered to investigate suspicious results. In Philadelphia, there were serious erasure issues over three years at about 50 schools.
The public has been told remarkably little about the findings of those investigations, even less outside of Philadelphia than in our district. We do know that in some cases, the suspicions of cheating were corroborated. After new test security procedures were implemented, the scores plummeted at many schools in Philadelphia and across the state. We do not know how much more valid the test scores are now than they were before the security changes. The state decided, because of cheating at some schools, that students and staff in all Philadelphia District schools must now be subject to different and stricter testing protocols from students in most districts. Their regular teachers can’t proctor the standardized test.
Jumping to the topic of increased high school graduation rates – they can’t be changed so easily with an eraser, but it’s important to be aware that in a high-stakes environment, different factors may be playing into the improvement we've seen here. Philadelphia has developed an innovative system of second-chance, accelerated high schools that are succeeding at re-engaging students who stop going to school. That’s one of a few interventions that certainly helped boost graduation rates. But there are also periodic stories about teachers being told not to fail students – do we know that this is not also a part of the increased graduation rate?
Data about school climate and safety are among the murkiest because of inconsistent reporting. The Notebook is generally reluctant to report these as if they have any validity as an indicator of school climate and safety.
In sum, the school performance indicators that are most widely used are too crude and inadequate for telling us whether more students are getting the high-quality education they need for post-secondary success.
What’s available now and may be a much more meaningful metric -- but still isn’t widely shared -- is data through the National Student Clearinghouse that can show us, by high school, whether students are enrolling in higher education, persisting to a second year, and graduating.
While we wait for that type of data about postsecondary success to become more readily available, we take the data that we have with many grains of salt and check multiple indicators because there are often inconsistencies. We've learned to be skeptical about a school that stands out on one indicator. An example: Mastery Charter Schools here has gotten kudos for its turnaround efforts, including boosting state test scores and graduation rates. But Mastery students’ SAT scores in reading and math hover below or around 400, 100 points lower than the national average, and well below the average for African American students.
What requires a whole other conversation is the other areas that aren’t well-measured or aren’t tested – writing, the arts, social studies, teamwork, leadership skills. It’s hard to say there’s meaningful improvement in Philadelphia schools without considering whether these important areas are even being seriously taught anymore.