Click Here
view counter

Scores are up again, except for ...

By Paul Socolar on Aug 13, 2009 08:37 PM

The School District presented a report to the SRC and the public on its latest test score results on August 12 and highlighted a seventh consecutive year of gains in both reading and math scores on the PSSA exam. However, Superintendent Ackerman focused her remarks on how far the District has to go and on the continuing racial achievement gaps, passing up an opportunity to applaud District teachers, whose union contract expires this month.

Proficiency rates this year, averaging about 3 percentage points better than last year in both reading and math overall, are moving clearly upward in most but not all categories.

One exception is the Tier I empowerment schools, which encompass 23 of the District's lowest-scoring schools (including eight neighborhood high schools). Test scores were essentially flat in this group, despite a major infusion of resources last year. Another apparent exception is 11th graders.

In previous years, the District has released grade-by-grade comparisons (see Chart 2) to the previous year's test scores, but this year they did not. This kind of selective release of test score data has long been a point of frustration for us Notebook reporters. When they leave out some chart they've provided in the past, usually it's covering up not-so-good news. When we pulled up last year's results for comparison purposes, we found that the scores were in fact up in every grade 3-8, but we discovered the results in grade 11 were essentially flat, a fact that was not reported in the presentation to the SRC.

SRC member Heidi Ramirez did ask a question about scores being worse in grade 11, and Superintendent Ackerman responded that students aren't taking the PSSA seriously - but will when it starts counting toward graduation.

Besides the dreadful grade 11 numbers, the other striking news in the grade-by-grade results is good news:  proficiency rates have cracked 60 percent for the first time in two categories: grade 8 reading (62% proficient) and grade 4 math (61% proficient). It's also the first year that more than half the students are proficient in math in every grade from 3 right through 8.

But the continued low scores in grade 11 have got to make you wonder "What does this all mean anyway?"

view counter

Comments (9)

Submitted by Mr.Boyle (not verified) on August 14, 2009 9:57 am

While sitting at the SRC meeting I was wondering what a reporter thinks of Wiener's/Ackerman's claim that at the current SDP rate of growth, our students won't be proficient until 2123. Is this a useful piece of information?

Submitted by Paul Socolar on August 14, 2009 11:00 am

Actually the claim got misreported.. What it says on the graph itself  is that the "Empowerment high schools," the District's lowest performing high schools, are on a trajectory where it would take til 2123 to reach 100% proficiency. The point is that at this subgroup of schools, the rate of improvement is extremely slow.

That's a legitimate point, however sensationalistically it's being presented (and of course who knows how proficiency will be defined in 2123).

What I have bigger questions about is the general charge that "we must accelerate improvement."  Obviously looking at performance levels in the District overall, and particularly at the dropout rates, we still have a system in crisis. But it's very uneven.

Nobody's said what would be an achievable but adequate level of gains.. If you look at 8th grade reading scores, proficiency rates are 62% and there was a 7% gain relative to last year. A graph could be drawn showing that at that rate our 8th graders would all be proficient in 2014. Cause for celebration?

Submitted by Paul Socolar on August 14, 2009 11:33 am

To elaborate on my post, it seems like the million dollar question is still how much these scores really tell us about whether students will graduate with the skills they need to succeed in post-secondary education and the workforce.

If we have such a big disconnect between performance in grades 3-8 and performance in grade 11 (proficiency rates still in the 30s), it's reasonable to assume that the rising scores in grades 3-8 haven't translated into big improvements in the education levels of the 57% of students who graduate.

Submitted by Philly High School Teacher (not verified) on August 15, 2009 12:55 pm

Contrary to Dr. Ackerman's assertion that 11th graders don't take the test seriously, the students I proctored this year took the reading and math tests seriously. Yes, when the tests impact graduation, it may affect some students but the low scores are not because of apathy. One factor which may influence the test scores, even slightly, is the testing endurance. 3rd - 8th graders take the test over 6 days - or one test a day. 11th graders have to take the test over 3 days - or two tests a day. Some students tire out.

That said and while my comments are only based on my observations versus data analysis, this year's 11th grade PSSA for reading seemed easier than previous years. The reading passages were very similar to the PDE released items. The constructed response (open ended) did not require much more than finding a main idea or author's point of view and giving two pieces of evidence from the reading. Students weren't asked to evaluate, infer, etc. nor apply the information / ideas. There is always a poem or two, a fictional passage along with non-fiction topical readings and usually a biography. (While I like poetry, I don't understand why it is so critical on the PSSA.)

The math was predominantly Algebra I with some geometry. The open ended are "real world" situations. If 11th graders had a year of Algebra I with some geometry in 11th grade, rather than Algebra II, some may have done better.

Nevertheless, is this the type of work students are expected to do in college? Maybe in a few introductory classes but certainly not much else. Some of the math may have more work place application. Most of the reading does not. The writing PSSA is certainly not what is expected of a freshman in college - it is an "easy" score. Even the open ended prompts in reading, which are based on a 3 point scale, don't take much to get a 2 out of 3. It is similar in math but it is based on a 4 point scale.

(The science and writing PSSA are treated as "step children" - no special roster and often pushed back until it is convenient. There also is little effort to make sure make-ups are done. Science and writing don't count for AYP).

Hopefully, the subject area tests which are suppose to be phased in will at least test what was suppose to be taught (e.g. Algebra I test for Algebra I, etc.) I think students will perform better than the generic PSSA. Nevertheless, the tension between curricula and testing which is suppose to be college prep versus work force prep will not be addressed. I'd like to think whether one goes to college or into a trade, one of the most important areas of study is civics. Not civics which is memorization or trivia but civic engagement and action which honors agency and action. That can't be measured in a multiple choice test. This is more applicable to the senior project but in Philly high schools, the senior project is often a second thought and what is accepted is often a joke. (I know there is a long term grant that is suppose to be working on the senior project but it is taking a long time to get it out to most of the high schools and who knows how many teachers have much input.)

If the move is to the subject area tests, then (hopefully) most students will complete the required tests (English, biology, algebra 1, history?) during 9th and 10th grade. That would leave more time during 11th and 12th grade for a senior project which is either college or work place prep (e.g. a genuine research paper with service learning) based on the student's interests. It will hopefully also give due to civics and provide time for students to specialize in content areas of their choice and possibly do an internship.

Submitted by Paul Socolar on August 15, 2009 1:00 pm

thanks for those very perceptive comments.  Sounds like you're suggesting the scores could be lower in 11th because of test fatigue and also worse alignment between the curriculum and the test. From talking to my daughters, who recently graduated, I do have a sense that 11th graders are somewhat more likely to blow off the PSSAs than younger students, even if that's not the predominant attitude.

But I still wonder whether those are adequate explanations for the growing gap between 8th grade and 11th grade test performance.  If you believe the PSSA results for grade 8, it seems like students are in fact coming out of 8th grade much better prepared academically than ever before, yet some key indicators of high school student performance (PSSA scores and SATs) have barely moved.  The graduation rate has moved significantly, but it seems like that's had a lot do with recapturing dropouts and credit accumulation strategies rather than raising overall performance.

So I'm still left wondering: unless the 8th grade PSSA scores are falsely optimistic, doesn't it seem like students must just be regressing academically at most high schools?

Submitted by Erika Owens (not verified) on August 15, 2009 3:00 pm

This could be related to the curriculum match up issue, but I wonder if by 11th grade the holes in what students have learned are just impossible to cover up.

I have the most experience working with math, and I could, and did, see students limp through math without understanding basic concepts. For example, maybe the student didn't learn borrowing across zero but could otherwise subtract ok. On a test about subtraction they might do ok and just miss all those borrowing questions. But once you get into more complex math seemingly small holes like that can completely trip you up. If you have to add, subtract, multiply, divide, and use fractions to solve an algebraic expression and you never mastered all of those concepts, you're probably in trouble!

I would imagine it also gets more difficult for teachers to target the problems when students get to more advanced level. Without paying a lot of attention to the student's work on how they got their answers the teacher might miss that borrowing issue and just think the student didn't get Algebra. But in elementary school if you are only doing subtraction, it would be much more obvious (though, clearly, it gets overlooked there too!).

This sort of issue would also result in the lower grade scores being overly optimistic. But if this sort of discrepancy between grades is sort of being glossed over then it sounds like there isn't the drive necessary to improve the 3-8 measures to pinpoint problems earlier instead of heralding increases that aren't sustained as progress.

Submitted by Philly High School Teacher (not verified) on August 16, 2009 9:33 pm

Thanks for your math examples. Brought back memories reviewing borrowing across zeros with my kids... Whether math or reading, since the skills are cumulative and math, in particular, is very dependent on applying many rules, it is difficult to "catch up" in high school. The reading gets complicated because of vocabulary and lack of familiarity with comprehension strategies, I have found, with non-fiction text. It is one thing for a student with a 6th grade reading level in 9th grade to move up a couple levels. It is another to move a student with a 3rd grade reading level in 9th grade anywhere near what is required for high school work.

Nevertheless, testing fatigue, apathy, lack of skills, etc. don't answer the 8th grade high to 11th grade slump. This is also compounded by the fact most students who drop out do so before 11th grade - they aren't there for the PSSA. (I agree that the increase in graduation rates is because of "credit accumulation strategies rather than raising overall performance." I know a number of students who graduated "just because.")

One of the stats that has was "buried" in the Inquirer was 42% of students miss at least 8 days per marking period. For high school, that is over 32 days a year. I'm sure the number for those students is closer to 50 or more days. That has to impact learning and it can not be blamed on the teacher. It is also not only a phenomenon in June - it is all year. Coming to school on time and ready to work is a work/college readiness skill that is not being addressed and is essential for much else to occur.

Submitted by f (not verified) on August 18, 2009 5:00 pm

Typically the mind tends to wander when presented with too much detail. Less is best when constructing a persuasive argument regarding any topic. The presentation of PSSA data information to the SRC last Wednesday by David Wiener and Dr. Ackerman was a good example of this axiom. There was just enough information in this power point presentation to convey the dual impression that some progress is being made by the students of the School District of Philadelphia but that this progress isn’t good enough.

Unfortunately the truth suffers when the information we are provided is at the superficial level that Mr. Wiener/Dr. Ackerman offered in this presentation. But then what is the truth? Dick Polman’s American Debate Blog entry of August 14, 2009 offers an interesting definition of truth.

“The truth is what people come to believe. The truth is message plus repetition.”

According to this definition, the truth that has been offered consistently during the seven years of NCLB school reform rhetoric is that public schools are failing our children. This message has been repeated over and over in the hope that we will eventually come to believe it. The school district’s data presentation at the last SRC meeting is just another repetition of this message.

A particularly good example of this form of truth telling is seen in the one slide presented by Mr. Wiener that showed the projected progress of eleventh grade empowerment high schools students towards the goal of 100% PSSA proficiency. Mr. Wiener suggested that academic proficiency for all eleventh graders will not occur until 2123 at their current rate of progress. This conclusion is based on his shockingly inappropriate and inaccurate use of the concept of statistical probability. Given the information utilized (the scatter plots of a small handful of prior years of eleventh grade test results) how could he or anyone draw the conclusion that he did? The graph on this slide communicated information that was a mile long and an inch deep. This type of data representation should be questioned and challenged.

In fact we are long over due for a careful consideration of the voluminous amount of superficial data that is offered up to the public as proof that our schools are failing our children. In this same presentation to the SRC, Mr. Wiener presented a comparison of the progress of all public school students to all empowerment school Tier One students. The conclusion of which was that the students in Tier One schools are not performing well. How useful is this comparison? Wouldn’t we be better able to get a sense of the progress of the Tier One School’s students if we compared them to a like socio-economic group of students who attend like schools that have made AYP? So why wasn’t this type of comparison offered to the SRC?

I don’t accept the point of view that truth is message plus repetition. So I wonder what is the truth concerning the performance of our public schools?

Submitted by Paul Socolar on August 18, 2009 7:32 pm

I'm not sure to what extent I'm saying the same thing, but I share your concern about superficial data. I think there is more sophisticated data analysis being done by the District and others, but for public consumption it feels like the data about student performance doesn't get much beyond three basic facts that have been recited for the past 7 years: 1) test scores keep going up in the elementary grades at most schools 2) most high school students still perform poorly and barely half graduate, and 3) there are still large gaps in student performance along racial lines. Any number of urban school districts are probably reciting the same three facts.  I'm not sure how much that story line really tells us about the performance of our schools or what we should be doing differently.

Post new comment

The content of this field is kept private and will not be shown publicly.

By using this service you agree not to post material that is obscene, harassing, defamatory, or otherwise objectionable. We reserve the right to delete or remove any material deemed to be in violation of this rule, and to ban anyone who violates this rule. Please see our "Terms of Usage" for more detail concerning your obligations as a user of this service. Reader comments are limited to 500 words. You are fully responsible for the content that you post.

Follow Us On

Philly Ed Feed

Recent Comments


Public School Notebook

699 Ranstead St.
Third Floor
Philadelphia, PA 19106
Phone: (215) 839-0082
Fax: (215) 238-2300

© Copyright 2013 The Philadelphia Public School Notebook. All Rights Reserved.
Terms of Usage and Privacy Policy