Look Out, NY Opt Out: Here Comes the Pro-Testing Charm Offensive

The University of the State of New York (USNY) has a new Commissioner of Education.  By a unanimous vote, the Board of Regents selected MaryEllen Elia, the recently fired superintendent of Hillsborough County, Florida,  to head the New York State Education Department (NYSED) and serve as President of USNY which, in addition to overseeing the entire public K-12 education system of 7000 schools, oversees more than 240 public and private universities, 7000 libraries, the state archives, special schools for the hearing and visually impaired, over 750,000 licensed professionals, and over 200,000 certified public school teachers.  She replaces former Commissioner, John King, Jr., and unlike her predecessor, she brings significant experience with public education, including a decade leading the 8th largest school district in the country where she was awarded 2015 Superintendent of the Year for Florida just a few weeks before a series of conflicts with the school boiled over in her early dismissal.  Under her leadership, her district was given a $100 million grant from the Bill and Melinda Gates Foundation to tie teacher evaluation and compensation to student performance.  While the grant program includes mentoring, principal evaluation, and peer evaluation, the district was also tasked to “develop fair and accurate measures of effective teaching” — for the Gates Foundation, this almost always means including at least some growth measures based upon student test scores.

Ms. Elia is certainly a change from former Commissioner Dr. John King Jr. whose impressive academic credentials were never matched with an equally impressive ability to listen to stakeholders and whose lack of experience at any level of public education was painfully obvious.  From her recent statements, Commissioner Elia is aware of what undid her predecessor:

“I think it is important for us to communicate with all of those people who have the stake in what’s happening in education,” said Elia, who most recently led the nation’s eighth largest school district, Hillsborough County, Florida, a racially and socioeconomically varied area that includes the city of Tampa. “So, yes, my plan is to be out in the state, listening to various groups and getting feedback and making sure that there is a response when that feedback is brought back to the department.”

Whether or not she is genuinely capable of do so remains to be seen.  Although she ran Hillsborough for an impressive ten years and was successful in securing the Gates Foundation grant, her removal represented long standing frustration with her leadership style which critics described as consistently uninterested in communicating with people she deemed as opponents.  More pronounced criticism described a workforce under Ms. Elia that was “cowed” and afraid to speak up about concerns for fear of retaliation, and board members complained they often did not get information they needed from her — even when a 7 yearpold stopped breathing and later died during a school bus ride.  Commissioner Elia had strong and loyal defenders as well, especially among the business community, but if her primary role coming back to New York is to lead a charm offensive that Dr. King was never able to do, watchdog organizations in the Empire State will need to keep a close eye on the substance behind the style.

While our new Commissioner is preparing to go on a speaking and listening tour of the state, she would do well to try to understand exactly why New York is the current leader in the nationwide Opt Out movement against today’s standardized testing policies, having seen test refusals jump from nearly 60,000 in 2014 to 200,000 in 2015.  In comments to the New York State Council of School Superintendents, Board of Regents Chancellor Dr. Merryl Tisch, lamented parents who opt their children out of standardized examinations, compared them to people refusing vaccination for their children, and pledged that “…we are going to continue to help students and parents understand that it is a terrible mistake to refuse the right to know.”  In April, Chancellor Tisch insinuated that the growth of the opt out movement was the fault of the dispute between New York Governor Andrew Cuomo and the state teachers’ union, making roughly 200,000 families pawns in a labor dispute.

So let’s just say that if Commissioner Elia is going to travel the state to understand the concerns of families and teachers, she needs to genuinely listen because NYSED has had cotton stuffed in its ears for some time now.

The first thing she needs to understand is that simply explaining why we test as suggested by Dr. Tisch is not going to be sufficient.  The still growing discontent in New York is not simply because nobody has bothered to explain the vision behind education policy in the state – to the degree that such a vision exists.  The reality that nobody at NYSED appears willing to examine is that parents understand that there are very real and actually tangible costs to making standardized testing as high stakes as it has become in the No Child Left Behind era, and, worse, they are increasingly aware that those policies do not work and should be set aside.  What has happened in the past decade and a half is a classic example of ever increasing perverse incentives that have taken standardized tests and converted them from an occasional check on the system into an increasingly important end unto themselves by which entire schools and individual teachers’ lives depend.  Since little has been done concurrent with high stakes accountability to actually support and improve schools with resources and innovative services, the result has been a policy environment where the tests have consumed more and more of the curriculum.  If you do not understand that parents are increasingly fed up with these phenomena and if you do not have a reasonable set of answers for them, then it is not likely that they will be swayed by mere explanations of why NYSED does what it does.  Parents want change, not platitudes.

It is unclear to me if Ms. Elias is suited for that task.

While New York’s new commissioner is clearly far more experienced and far more understanding of how education consists of intersecting and overlapping stakeholders that policy must consider, her record is no less devoted to the core elements of “reform” — Common Core Standards, standardized testing, use of testing to rank and sort schools and teachers — than her predecessor’s or her new Chancellor’s.  In the application for the $100 million grant from the Bill and Melinda Gates Foundation, Hillsborough predicted they would fire at least 5% of the districts tenured teachers for “poor performance,”  and the grant work led her to develop, with collaboration from the teachers’ union, an evaluation system that uses test scores for 40% of teachers’ ratings.

All but guaranteeing the percentage of teachers you will fire in an application to revamp your teacher evaluation and reward system should raise any serious thinker’s eyebrows.  It smacks of the kind of stack ranking of employees that, ironically, the Gates founded Microsoft finally ditched after a disastrous decade of evaluating employees that way destroyed effective collaboration.   If the Hillsborough application was taken seriously in the early years, teachers with low growth scores had to be constantly concerned if they would hit that bottom 5% in combination with other measures and be in danger of losing their jobs.  While not as daft as the Microsoft system that required every employee in every unit to be placed on a normal curve, the five percent prediction amounted to over 420 teachers a year.  As it turns out, the district came nowhere near that number by 2012, but it did manage to make a significant number of employees jittery.

Of greater concern is Commissioner Elia’s comments on how to incorporate test scores into evaluations as she enters a state with a new evaluation matrix that gives those scores an entire axis:

“The research is very unclear on any weight at all,” she said, when asked about Governor Andrew Cuomo’s proposal to base evaluations 50 percent on tests. “There have not been any studies that indicate that 50 percent is better than 40 percent is better than 20 or 30. And so I think what we need to do is get out there, work together collaboratively to come up with what we believe is a reasonable approach to evaluation, and constantly be getting feedback. And when it needs to shift, we need to shift it.”

I’d like to offer a suggestion on what weight to give standardized test scores in the evaluation of teachers:

None. Zip. Nada. Bupkas.

The destructive nature of including standardized testing data in teacher evaluation is discussed above.  It narrows the curriculum.  It incentivizes schools and teachers to make the test itself the curriculum. It consumes instructional time and resources that could be better used.  It focuses learning on the least interesting skills and diminishes actual love of learning.  It serves as a disincentive for both teachers and students to take risks that might diminish test scores.  But there is an even more important reason to reduce the role of standardized testing data in teacher evaluation.

It doesn’t work.

Maybe one could have pretended otherwise in 2009-2010, but this should not even be controversial anymore.  Growth models for teacher evaluation based upon standardized testing data do not work.  In order for a growth measure to work, it has to be be able to peel away every factor that accounts for the differences among student test scores that is not attributable to the teacher, and we simply do not have statistical models that do this reliably.  Commonly used models have standard errors as high as 36% for a single year of data, and they would require a decade of data to reduce the likelihood of mislabeling a teacher to 12%.  Growth models are unstable, and ones that tend to produce stable results tend to be poorly designed. The models have a strange ability to label even teachers who are locally known to be excellent working with advanced students as ineffective because of how little room there is for students to not hit the model’s predicted scores.

No wonder then that the American Statistical Association released a statement in 2014 saying that Value Added Models should not be use for teacher evaluation.  Yet here we are in 2015 with Governor Cuomo having successfully browbeaten the state Assembly and Senate into passing a budget that makes value added measures based on test scores effectively half of the evaluation system for teachers, and with a new Commissioner who is pondering what percentage is “correct” for such measures. This all but guarantees that the tests will continue to have both a disruptive and distorting effect on schools and classrooms, threatening teachers who are good at what they do and diminishing the depth and breadth of the curriculum students experience.

It also means that the reasons for the Opt Out movement to both exist and grow remain firmly in place.

Education reformers today seem to treat any resistance to their favored policies as simple matters of marketing — throw a lot of money at consistent messaging and people will come around to realize that they actually love what you are selling.  That approach can work in the world of innovative technology where people need to learn how it can change their daily lives. Education reform is not like that, however.  First, we are pretty familiar with how standardized testing is overwhelming education as we well into the second decade of test based accountability.  Second, people do not favor using those tests to evaluate teachers; while over 60% strongly agree that evaluation should help remove ineffective teachers, 61% oppose using tests scores to do that, up from 47% in 2012. Third, in the same PDK/Gallup Poll, parents with children in school reported something they have consistently said over decades: they like the schools their children attend. For 30 years, the percentage of parents giving their children’s schools grades of A or B has hovered near or above 70%.  It has dipped lately, but that is as likely connected to the disruptive impacts of Common Core and associated testing as it is connected to parents agreeing with reformers.

So reformers may want to believe they need to sell families on a new iPhone.  In reality, they are peddling New Coke: messing fundamentally with something people like without giving them a substantial benefit in return.

This is the challenge Commissioner Elia faces as she considers how to mount a defense of New York state policy to an increasingly restive population.  If she continues to try to convince parents that they really love the taste of New Coke instead of laying the groundwork for the NYSED to walk back its disastrous policies, this will not go well.