You may have missed that National Council for Teacher Quality (NCTQ) released another report recently, this one taking aim at teacher absenteeism. It being an NCTQ report, it is not an exceptional work of scholarship, and they try to discuss policy implications absent any real statistical analysis. This is par for the course from an organization that purports to seriously investigate the quality of teacher preparation in this country, but which does so by sitting in their offices looking at course descriptions online and calling up schools of education demanding materials and, failing that, trying to deceive schools into giving them those materials. Needless to say, their errors are both shocking and laughable (my personal favorite is that Columbia got recognition for how selective its undergraduate teacher preparation programs are — those programs don’t exist, so they are VERY selective), but they are less laughable when you realize that influential people take them at face value. It just goes to show that sometimes all you need to do is to put the letter “N” in your acronym to be considered serious.
This current report is perhaps not as egregious, but it is also not exceptionally interesting, mainly because all NCTQ does is provide a national overview of descriptive statistics regarding teacher attendance and then claim that they have proven something. Two problems: First, the descriptive statistics are not complete. NCTQ spends most of the report providing bar and pie graphs and a lot of averages, but they do not appear to have calculated means, medians, modes or standard deviations. Consider that they divide teachers into attendance categories of “excellent”, “moderate”, “frequently absent” and “chronically absent”. How many standard deviations separate a “chronically absent” teacher from a teacher with “excellent” attendance? Damned if I know, and NCTQ provides no indication that they know either.
Second, NCTQ tries to draw conclusions from these data without demonstrating that they have analyzed them thoroughly using statistical inference. For example, contrary to previous research from academics at Harvard and Duke, NCTQ claims there is no connection between the poverty characteristics of a district and teacher absenteeism. In fact, they said:
Given the existing research on teacher attendance, an increase in teacher absenteeism was expected as school poverty levels increased. Surprisingly, there was no significant increase in these districts. The difference between the average days absent in the highest and lowest poverty schools was under one day and was not statistically significant.
How did they prove that? Well, from the report, they lined up some bar graphs, and that appears to be it. The report appendices claim use of a significance test that did not apply to poverty characteristics and a use of a variance test that did, but these results are not provided in the text of the report. Readers are, supposedly, to look at the bar graphs, nod their heads and agree. Such appeals to “obviousness” are a good way of avoiding doing sophisticated analysis (NCTQ does not mention any statistical testing that uses sophisticated methods capable of capturing poverty effects), but they are also a way of thoroughly deceiving lay people with no knowledge of statistical reasoning.
Such a lay person turned up on my Twitter feed. Looking for any news on the NCTQ report, I came across this tweet from a Mr. Bob Bowdon:
Not knowing who he was, I replied:
I thought that was it, but a few days later, he responded:
This is a typical strategy when given a reference that poses serious questions about something: take a little bit of it and use that to defend what you like against the rest of the article. So I gave it another try:
I decided at this point to figure out who @BobBowdon actually is, and in short order found out that he is the director and producer of the “fauxcumentary” film “The Cartel” wherein he takes a hard look at education in New Jersey and decides that everything that is wrong with our schools can be traced back to and directly blamed upon…you guessed it: teachers’ unions. He is a practitioner of a form of “advocacy journalism” that looks like what would happen if you blended Michelle Rhee and James O’Keefe. Unsurprisingly, his work is riddled with errors in assumption and fact, points amusingly documented by Rutgers Professor Bruce Baker, here, here, here, and here. I suppose he was feeling confident given that the NCTQ report came with numbers and graphs, so he replied again:
And I replied:
What I got back seemed defensive (and clueless):
Trying to be nice one last time, I responded:
There’s a stance in his comment that I find fascinating, however. It is one that is impervious to even considering a critique and relies heavily on ignorance to bolster a position that is not especially strong. I have no idea if Mr. Bowdon has no working knowledge of the difference between descriptive and inferential statistics, but he displays no interest in finding out here. Further, he retreats into calling my objections “convenient” rather than inquiring after their substance.
His next tweet:
Ah, yes, “apologists” – boilerplate anti-union talk wrapped up in a thorough lack of understanding of how research is conducted whether deliberate or otherwise. His complaint here is instructive, however. Saying “any study can be called ‘incomplete'” covers the fact that the NCTQ report can barely be called a “study” because it really has no research questions but it tries to make inferences from the incomplete descriptive statistics that it employs. Calling teacher “absences” “hard objective stats” is, I suppose, some attempt to claim there is no ambiguity in the data and any inferences we want to make from them can be made by appealing to “obviousness”. This is how someone thinks when they truly do believe that you can make “statistics say anything” but it not anything that would be said by any honest person who understands reasoning with statistics. For example, the NCTQ data suggests that teachers are absent more than the workforce average, and from that, they infer a need for policy interventions. But without knowing the REASONS for teachers being absent, or if such absence rates are actually significant, it is impossible to make any informed comments on existing or possible interventions. What if teachers are absent more than average because they work in close contact with children and get sick more often than the rest of the workforce? What if teachers are more absent because the teacher workforce is still largely female and women still bear a disproportionate share of child care duties in American families? Policy interventions for either of these circumstances would be radically different, but Mr. Bowdon is obviously only interested in blaming teachers specifically and unions generally.
So I took out the teacher in me:
Okay, fine, that’s a lot to read, but I was surprised that he tried to as if I hadn’t just written to him like he was capable of understanding my points:
I admit that baffled me. But I have been teaching for twenty years, and I think that even the most obstinate of students is capable of a breakthrough:
Mr. Bowdon was still pretty oblivious:
Sometimes you have to throw in the towel:
Mr. Bob Bowdon is not an isolated case of low information punditry trying to stake out ground as a “thought leader” in education. He calls his website a national “hub” for online information about education reform, but his bias in that space is obvious, where he actively champions education “reformers” seeking to increase testing, spread charter schools and curtail teacher unions and where he labels people who oppose such efforts as the education “establishment”. Mr. Bowdon finds the NCTQ report important not because it reveals interesting and important insights into the teacher workforce (it doesn’t), but because he can use it to argue that unionized teachers abuse the “perks” of their employment. This is similar to no end of billionaire-funded efforts to fundamentally change the way we offer schools without a public vote and which dare to call themselves civil rights movements.
It may be momentarily fun to demonstrate how low knowledge many foot soldiers for corporate reform are, but it is also entirely serious.
Daniel Katz, Ph.D. 