Arne Duncan’s Great Kid Story Problem

In his speech laying out administration priorities for the renewal or rewrite of No Child Left Behind, Secretary of Education Arne Duncan turned to a personal anecdote to explain the imperative of accountability based reform:

In between my junior and senior year at college, I took a year off to help in my mother’s after-school tutoring program on the South Side of Chicago and figure out if I really wanted to devote my life to this fight for educational opportunity.

One of the students I tutored was a basketball player at the local high school, who was studying to take his ACT.

He was a great kid who had done all the right things. In a very violent neighborhood, he had stayed away from the gangs. He didn’t drink, he didn’t use drugs. He was actually an honor roll student with a “B” average, and on track to graduate. I initially thought this was absolutely a young man who could beat the odds and defy the negative stereotypes of young black men.

But as we started to work together, I was heartbroken to quickly realize that he was basically functionally illiterate.

He was reading at maybe a 2nd or 3rd grade level, and was unable to put together a written paragraph. Tragically, he had played by all the rules, but had no idea how far behind he was. Throughout his life, he had been led to believe that he was on-track for college success.

And he was nowhere close.

The educational system had failed him, and the buck stopped nowhere.

This is the kind of personal story that makes great fodder for satirical pieces in The Onion, but I will grant Secretary Duncan a point: there are children in the system who are passed along from grade to grade without learning enough to be successful in more complex subjects later on.  And the Secretary has a point that in too many cases like these few people are willing to take responsibility.  When I talk to my education students about this, I frame this as a cycle of blame passing:  The ninth grade teacher has a student who cannot write well and blames the junior high teachers for what they didn’t teach.  The junior high teachers blame the middle school teachers, and the middle school teachers blame the elementary school teachers.  Eventually, the child is in utero and nobody has taken proper responsibility for teaching the child as he has arrived in the classroom that year.  Given that higher education institutions report that about 20% of first year students need to take at least one remedial class when they arrive (and even 12.8% of entering students at very selective 4 year schools), it is reasonable to ask if our elementary and secondary education systems can do a better job preparing more students for further schooling.

Of course, answering such questions are complex.  Critics and reformers often point to the number of college students in need of some remediation and state those students are “not ready” for college.  That’s far too broad a brush.  For starters, the numbers are variable by the type of institution reporting, by race and ethnicity, by gender, by age of student, by dependency status, and by the educational obtainment of the parents of the student receiving remediation.   Additionally, students can receive a wide variety of remediation in college from a single studies skills course to an entire plate of courses meant to “plug holes” from elementary and secondary education.  A student who dropped out of high school, got a GED at 25, and enrolled in Community College who needs math instruction to progress in a STEM program is far less worrisome than the young man in Secretary Duncan’s anecdote who is reported as laboring under the impression that his reading level being at The Magic Treehouse series is going to get him into college.

There’s just a problem.  Secretary Duncan’s priorities for the NCLB revision won’t help him either.

It isn’t that someone shouldn’t have taken responsibility for the young man’s learning (although how Secretary Duncan, at the callow age of 20 or 21 could actually tell that nobody had done so is left unexplained); it’s that forcing that responsibility by holding his teachers accountable to his standardized test scores each and every year, as favored by the Obama administration, is one of the worst paths to take to help him.

“Testing” is not a dirty word.  As part of a multiple assessment system to help teachers, students, and parents know where students stand and in what areas students need help.  Formative assessments, however they are developed and administered, are meant to provide the kind of feedback that can personalize instruction and help teachers as they create a rich and complete curriculum.  Dr. Bruce Baker of Rutgers University notes for what such assessments cannot be used:

This information should NOT be used for “accountability” purposes. It should NOT be mined/aggregated/modeled to determine at high level whether institutions or individuals are “doing their jobs,” or for closing schools and firing teachers. That’s not to say, however, that there might not be some use for institutions (schools districts) mining these data to determine how student progress is being made on certain concepts/skills across schools, in order to identify, strengths and weaknesses. In other words, for thoughtful data informed management. Current annual assessments aren’t particularly useful for “data informed” leadership either. But this stuff could be, given the right modeling tools.

This is the approach we use to ensure that no child is left behind. By the time annual, uniform, standardized assessment data are returned in relatively meaningless aggregate scores to the front office 6 months down the road, those kids have already been left behind, and the information provided isn’t even sufficiently fine grained as to be helpful in helping them to catch up.

Dr. Baker differentiates testing used for individual diagnostic purposes and testing used for accountability/system monitoring purposes:

When it comes to testing for system monitoring, where we are looking at institutions and systems, rather than individuals, immediate feedback is less important. Time intervals can be longer, because institutional change occurs over the long haul, not from just this year, to next. Further, we want our sampling – our measurements – to be as minimally intrusive as possible – both in terms of the number of times we take those measurements, and in terms of the number of measurements we take at any one time. In part, we want measurement for accountability purposes to be non-intrusive so that teachers and local administrators, and the kids especially, can get on with their day – with their learning – development of knowledge and skills.

So, when it comes to “System Monitoring” the most appropriate approach is to use a sampling scheme that is minimally sufficient to capture, at point in time, achievement levels of kids in any given school or district (Institution). You don’t have to test every kid in a school to know how kids in that school are doing. You don’t have to have any one kid take an entire test, if you creatively distribute relevant test items across appropriately sampled kids. Using sampling methods like those used in the National Assessment of Educational Progress can go a long way toward reducing the intrusiveness of testing while providing potentially more valid estimates of institutional performance (how well schools and districts are doing).

The distinctions here should be obvious, and they are crucial:  accountability relies upon system wide data that is best captured via sampling, and monitoring system wide trends via data does not require that every child be tested in the same standardized test every single year.  As Dr. Baker has shown previously, trying to take this data and use it for accountability of individual teachers based upon value-added modeling does not produce results that are stable and are therefore pretty useless.

Wouldn’t you know that Secretary Duncan has it exactly backwards?

By insisting that large standardized measures be given to every child every year AND endorsing using those data for individual teacher accountability, the Secretary is calling for maintaining a standardized testing regime that is needlessly intrusive and for applying the data from those tests for the wrong purposes.  Worse, it is incentivizing the worst kind of teaching, practices to which Secretary Duncan gave a passing acknowledgement as destructive but which his insistence upon placing the highest stakes on an intrusive testing schedule will entrench into classrooms.  We’ve seen this in the years since NCLB with narrowing curricula and more focus on tested subjects that upon a full, rich curriculum.

One other rationale is possible by insisting upon annual, large scale examinations for every child, but it is one that betrays a lack of imagination.  Secretary Duncan said that:

I believe parents, and teachers, and students have both the right and the absolute need to know how much progress all students are making each year towards college- and career-readiness. The reality of unexpected, crushing disappointments, about the actual lack of college preparedness cannot continue to happen to hard working 16- and 17-year olds – it is not fair to them, and it is simply too late. Those days must be over.

That means that all students need to take annual, statewide assessments that are aligned to their teacher’s classroom instruction in reading and math in grades 3 through 8, and once in high school.

Secretary Duncan is suggesting that mass standardized tests given annually are the tools needed by parents to monitor their children’s progress.  I suppose there may be families out there who are itching for that packet from the state DOE that comes weeks or months after the standardized exam, but I think it is far more likely that parents would like to know that teachers have access and utilize a steady stream of tools to assist their students and to communicate with families.  As it stands, Secretary Duncan insists on giving those parents a single test result that can suggest something is going on but which cannot say a blessed thing about why it is going on.

Arne Duncan is worried about that great kid he met three decades ago.  Sadly, he doesn’t have a clue about what would have helped that child not get lost in the system.