Category Archives: Data

Welcoming a New Generation of Teachers

My university welcomes the Class of 2018 this week which means that I will begin teaching a new class of first year students enrolled in our secondary education and secondary/special education programs.  It goes without saying that I am consistently impressed with the caliber of young person I meet each year.  They have committed themselves to a program requiring hard work from them early in their college careers, and they have committed their talents and futures to a profession that is intellectually and emotionally demanding.  These are the types of young people I have admired since I began my work in teacher education in 1997 at the beginning of graduate school, and it is genuinely exciting to know how many of them over the years have stayed in teaching, honing their craft, becoming leaders and teaching many 1000s of young people over the years. This is incredible work.

My first year students were born in 1996, when I was still a high school English teacher, and they began Kindergarten in 2001.  This means that among the myriad of things the media likes to remind us that Millennials have “never known”, this class of Millennials has never known a school system without the Elementary and Secondary Education Reauthorization of 2001, popularly known as the No Child Left Behind Act (NCLB).  Hailed by President George W. Bush as refusing “the soft bigotry of low expectations,” NCLB ushered in an age when school districts, schools and teachers were to be held accountable by student results on mass standardized tests.  While President Barack Obama’s “Race to the Top” (RTTT) program was billed as loosening the punitive measures of NCLB, it has further entrenched mass test-based accountability by pushing states to adopt common standards and to include the results of students’ standardized test scores into teacher evaluation.  Any current hot potato issue in elementary and secondary education, from the Common Core State Standards, to the mass standardized testing and the use of those tests to evaluate can be traced back to the premise of both of these laws:  accountability of schools for students’ annual “progress” on mass testing is an appropriate lever to effect positive school change.

The cumulative impacts of these reforms on teachers, teacher morale and schools is a subject for another blog, but suffice to say that despite recent efforts to paint the picture more rosily, overall teacher morale has suffered and has suffered more in our schools that need help the most.  It hardly helps that most high profile efforts to “improve” teaching focus solely on weeding out teachers deemed to be ineffective and placing pressure on all teachers to demonstrate effectiveness via standardized test scores.  Absent in those reforms?  Improving school working conditions, increasing teacher collaboration and leadership, emphasis on markers of student learning and accomplishment outside of mass testing, addressing community poverty impacts and looking at what opportunities actually exist in our economy.

Despite all of this, I will meet a group of young people who want to teach.  Experience tells me that all of them, despite the environment in which they grew up, believe in the transformative potential of education and are genuinely committed to inspiring future generations of students.  

But this is also where a cautionary note must be sounded.  The process of becoming a teacher is not one that actually begins with university classes.  Most people begin to make the commitment to teach many years earlier.  Talk to an elementary school teacher, and you will frequently find someone who began with make believe games set in an imaginary classroom.  Talk to a secondary school teacher, and you will often find someone whose love of subject matter set her apart from peers from middle school forward.  During their long “careers” as K-12 students, future teachers observe upwards of 15,000 hours of teachers teaching which forms the backbone of what Dan Lortie called “the apprenticeship of observation” with which all teachers enter their formal preparation.  Unlike professionals in medicine and law, most students of teaching are intimately familiar with being the recipients of teachers’ practice, and it is that familiarity that largely inspires them to enter the field and informs their deeply personal visions of what it means to teach.

Many researchers have noted to much of what future teachers learn from this apprenticeship is incomplete and fails to capture all of the work that goes on beyond teachers’ in classroom performances.  Regardless, it is a beginning, and an important one to people who want to teach — it is our job in teacher education to layer upon it, making elements of it problematic so they can be revised and adding to it the hidden pedagogical skills of teachers that are not generally learned before teacher education.

If learning to teach, if the very commitment to learning to teach begins with the process of one’s own K-12 education, then it is vitally important to the profession and its future that we are mindful of the kinds of schools in which the future’s teachers are currently enrolled.  I would argue that we have done a poor job historically, but especially in the past 15 years, of listening to what teachers themselves believe will help them be better at their profession.  According to Francie Alexander of Scholastic, INC., a survey conducted for a joint Scholastic-Gates Foundation study by the Harrison Group found the following

  1. Most teachers feel heard in their own schools, but 69% do not believe they are listened to by district, state and federal players.
  2. 71% believe they need more time to study and understand the Common Core State Standards before implementing them.
  3. Teachers value collaboration, but 51% cite a lack of time for collaboration as a challenge.
  4. 99% of teachers believe their work goes beyond academics.
  5. 88% of teachers believe the rewards of teaching outweigh the challenges.

While that survey cited high levels of teachers “enthusiastic” about the Common Core standards, more recent surveys have shown significant cratering in teacher support.  Further, the overall satisfaction reported in this survey has to be weighed in contrast with the 2013 findings of the 29th annual Metlife Survey of Teachers which found only 39% of teachers said they were “very satisfied”.

There is a lot of “churn” in the waters of education today, and it is beyond admirable that so many teachers are able to take professional satisfaction in the concept of the “small victories” many of them routinely see in their work with students and community.  It is equally admirable that young people with exceptional talents and skills seek to join the profession.

But we must be careful that reforms are not allowed to alter the aspects of schooling that make it such rewarding work.  Mass test-based accountability that reduces teachers’ work to an “effectiveness rating” tied primarily to test scores is a toxic approach.  Not only does it disrespect the fullness of the work teachers know that they do, but also it over emphasizes what can even been learned from such tests, and few current reform advocates put their efforts behind better support, collaboration and leadership.  Schools must remain humane places where teachers and students can meet as far more than average annual progress calculations, or we will lose those who wish to become teachers because they want to do good in the world.  If our vision of school tilts too heavily towards the technical/rational aspects of measurement in learning and ignores the humanistic development side, we will end up with future teachers who lack a rich and full vision of their profession.

Think of it this way:  If you have a baby born this year, she will be ready to enter high school in 2028.  Many of her potential ninth grade teachers were born in 2006 and are beginning 3rd grade this Fall, the grade where most high stakes testing begins in earnest.

What kinds of school experience do you want your child’s teachers to have?

Leave a comment

Filed under Common Core, Data, teacher learning, teaching, Testing, Uncategorized

David Berliner Responds to Economists Who Discount Role of Child Poverty via Diane Ravitch

From Diane Ravitch’s blog — incredibly important response to slight of hand “research”:

 

David Berliner Responds to Economists Who Discount Role of Child Poverty.

Leave a comment

Filed under Data, politics, Shared Posts, Social Justice

Jamming Square Pegs into Round Holes: Arne Duncan Sets Sights on Special Education

United States Secretary of Education Arne Duncan announced a new focus on special education on Tuesday of this week.  The federal government will shift its resources for monitoring state compliance with the Individual’s with Disabilities in Education Act (IDEA) from examining procedural compliance and begin looking at “outcomes” for students with disabilities using a new framework called “Results-Driven Accountability (RDA).  This new framework will include participation in state curriculum assessments and data on reading and mathematics achievement for disabled students using the National Assessment of Educational Progress (NAEP) the examination sponsored by the DOE every two years to gather a snapshot of national trends in education.  According to the Washington Post:

To calculate how states stack up under the new criteria, the department is using a complex matrix that weighs several factors, including how well students with disabilities perform on the National Assessment of Educational Progress, or NAEP, a test the federal government gives to a sampling of students in every state every two years.

NAEP is designed to offer a snapshot of academic performance. This marks the first time the government has tied NAEP scores to consequences.

Duncan brushed aside the suggestion that the new approach adds to a climate of high-stakes standardized testing. “I wouldn’t call it high-stakes,” he said.

Given that the federal government allocates 11.5 billion dollars a year to the states to assist with special education, that assurance is likely to ring hollow to state and local officials charged with compliance.

I will give Secretary Duncan credit for one factual observation in his conference call with reporters;  most students who qualify for an individualized education plan (IEP) do not have cognitive disabilities that severely limit their ability to engage with a challenging curriculum.  But pretty much every other underlying assumption of this shift to an RDA compliance system is problematic.

Let’s start with the existing compliance of states under previous federal guidelines.  The DOE notes that under previous compliance guidelines, 38 states were in compliance with IDEA and under the new guidelines that number will drop to 15. I would suggest that if previous compliance standards which focused upon procedural compliance told the federal DOE that 38 states were in compliance with no need of assistance or intervention then those procedural guidelines were hideously flawed.  The Bay Area NBC station found over 10,000 families in California went to court over disputes with districts over special education services, and that number represents only the fraction of families that had the resources to pursue their dispute to that level.  Even Massachusetts, a state that pioneered services for disabled students and which meets the new requirements, is not precisely immune from being sued for noncompliance.  While states are rated on their compliance, it is up to actual districts and schools to implement the provisions of special education law, and many districts, suffering from budget restraints and state aid cuts, have to be sued in order to even begin an evaluation process for a potential special needs learner. Secretary Duncan made some deal about the DOE’s 11.5 billion dollar commitment to special education in the country, but with 6.5 million students eligible for services, that amounts to an underwhelming $1,769.23 per student nationwide, and the Council of Exceptional Children (CEC) notes that in 40 years, the federal government has NEVER fulfilled its promise to fully fund IDEA.

If Secretary Duncan wants to improve services for special education students, he could start by endorsing full funding of IDEA and actually determine if states are even in procedural compliance with far better measures than currently employed.

Another flaw of this plan resides directly in the use of state assessments and the NAEP for purposes of assessing state compliance and, eventually, adding punitive measures for states whose disabled students do not make regular improvement on those exams.  Placing this requirement on the NAEP would be tremendous mistake for several reasons.  NAEP is designed to provide a snapshot of the educational landscape in the United States, and part of its usefulness is tied the lack of any significant stakes attached to it.  By potentially tying special education compliance to the NAEP, the incentive will exist for states and districts to make special education students’ education consist of test preparation.  Mr. Duncan can breezily dismiss that concern all he wants, but the best way to assure that special education students across the students find themselves in self contained classrooms aimed at test preparation is to measure compliance this way.  We have some idea about how this unfolds from NCLB already.

Secretary Duncan also made major mistakes in his assessment of special education students’ “rising to the challenge.”  I must emphasize again that he is partially correct:  classified students CAN and, in fact, DO achieve within materials similar to or identical to their general education peers.  Very few of the students who qualify for IEPs under federal law are significantly cognitively disabled, and it is an article of faith among professional teachers that “all children can learn”.

BUT — that article of faith comes with an important caveat: All children can learn to the degree of their ability when provided with appropriate accommodations and when measured in a manner that allows them to demonstrate their understanding.  In a way, this corollary applies to all children, but general education students are more likely to cluster around a set of skills and capacities that distribute normally on a standardized examination.  By definition, many students with disabilities do not, and this does not mean that they are incapable of learning.

It means we are often incapable of measuring their learning in a fair and accurate way via a paper and pencil standardized test.

This does not require a lot of imagination.  Picture a child with severe dyslexia or ADHD.  This is certainly a child who is capable of learning, and a skilled general education teacher working with a child study team and following a well designed IEP can create assessments of learning and supplemental experiences in the classroom where that child demonstrates substantial learning.  That same learning may not be on display during a paper and pencil standardized examination that requires hours of time in a seat.  This can apply to a child with sensory issues or a behavioral disorder.  It is not that schools should or do abandon such a child to not learn within the goals of a general education curriculum: it is that the entire process of special education is meant to serve accommodations that allow the child to engage the material and demonstrate learning in appropriate ways with input from experts on learning.

Mr. Duncan, do you have a standardized exam that does this?

But this is the problem with the federal DOE under Secretary Duncan.  Having committed to big data sets as the be all and end all of understanding what is going on in education and having determined that standardized test scores are the most important measure of educational accomplishment, we now have a special education compliance policy that is going to try to force the most square of pegs into Secretary Duncan’s round hole of test based accountability.

Image from Toothpaste for Dinner: http://toothpastefordinner.com/archives/2011/Jun/

Image from Toothpaste for Dinner: http://toothpastefordinner.com/archives/2011/Jun/

Before ramming 6.5 million special education students into test based accountability, I would suggest several alternative approaches:

1) Vigorously advocate for the CEC’s proposal to FULLY fund the Individuals with Disabilities in Education Act.

2) Monitor GENUINE procedural compliance with the provisions of IDEA.

3) Add new compliance measures such as parental satisfaction surveys with special education services provided.

4) Assist states with creation of qualitative measures of special education students’ progress.

5) Add federal assistance to community agencies that help connect families in poverty to special education services

NAEP data can remain what it ought to be: a snapshot of student skills that can inform the creation of further policy, but not be linked to consequences.

Leave a comment

Filed under Data, politics, schools, teaching, Testing

My Argument With the Director/Producer of the Fauxcumentary “The Cartel”

You may have missed that National Council for Teacher Quality (NCTQ) released another report recently, this one taking aim at teacher absenteeism.  It being an NCTQ report, it is not an exceptional work of scholarship, and they try to discuss policy implications absent any real statistical analysis.  This is par for the course from an organization that purports to seriously investigate the quality of teacher preparation in this country, but which does so by sitting in their offices looking at course descriptions online and calling up schools of education demanding materials and, failing that, trying to deceive schools into giving them those materials.  Needless to say, their errors are both shocking and laughable (my personal favorite is that Columbia got recognition for how selective its undergraduate teacher preparation programs are — those programs don’t exist, so they are VERY selective), but they are less laughable when you realize that influential people take them at face value.  It just goes to show that sometimes all you need to do is to put the letter “N” in your acronym to be considered serious.

This current report is perhaps not as egregious, but it is also not exceptionally interesting, mainly because all NCTQ does is provide a national overview of descriptive statistics  regarding teacher attendance and then claim that they have proven something.  Two problems:  First, the descriptive statistics are not complete.  NCTQ spends most of the report providing bar and pie graphs and a lot of averages, but they do not appear to have calculated means, medians, modes or standard deviations.  Consider that they divide teachers into attendance categories of “excellent”, “moderate”, “frequently absent” and “chronically absent”.  How many standard deviations separate a “chronically absent” teacher from a teacher with “excellent” attendance?  Damned if I know, and NCTQ provides no indication that they know either.

Second, NCTQ tries to draw conclusions from these data without demonstrating that they have analyzed them thoroughly using statistical inference.  For example, contrary to previous research from academics at Harvard and Duke, NCTQ claims there is no connection between the poverty characteristics of a district and teacher absenteeism.  In fact, they said:

Given the existing research on teacher attendance, an increase in teacher absenteeism was expected as school poverty levels increased. Surprisingly, there was no significant increase in these districts. The difference between the average days absent in the highest and lowest poverty schools was under one day and was not statistically significant.

How did they prove that?  Well, from the report, they lined up some bar graphs, and that appears to be it.  The report appendices claim use of a significance test that did not apply to poverty characteristics and a use of a variance test that did, but these results are not provided in the text of the report.  Readers are, supposedly, to look at the bar graphs, nod their heads and agree.  Such appeals to “obviousness” are a good way of avoiding doing sophisticated analysis (NCTQ does not mention any statistical testing that uses sophisticated methods capable of capturing poverty effects), but they are also a way of thoroughly deceiving lay people with no knowledge of statistical reasoning.

Such a lay person turned up on my Twitter feed. Looking for any news on the NCTQ report, I came across this tweet from a Mr. Bob Bowdon:

Not knowing who he was, I replied:

https://twitter.com/Prof_Katz/status/475703503910748161

I thought that was it, but a few days later, he responded:

This is a typical strategy when given a reference that poses serious questions about something: take a little bit of it and use that to defend what you like against the rest of the article.  So I gave it another try:

https://twitter.com/Prof_Katz/status/476391864489095169

https://twitter.com/Prof_Katz/status/476392014829744129

https://twitter.com/Prof_Katz/status/476395907668119552

I decided at this point to figure out who @BobBowdon actually is, and in short order found out that he is the director and producer of the “fauxcumentary” film “The Cartel” wherein he takes a hard look at education in New Jersey and decides that everything that is wrong with our schools can be traced back to and directly blamed upon…you guessed it: teachers’ unions.  He is a practitioner of a form of “advocacy journalism” that looks like what would happen if you blended Michelle Rhee and James O’Keefe.  Unsurprisingly, his work is riddled with errors in assumption and fact, points amusingly documented by Rutgers Professor Bruce Baker, here, here, here, and here.  I suppose he was feeling confident given that the NCTQ report came with numbers and graphs, so he replied again:

And I replied:

https://twitter.com/Prof_Katz/status/476396060986703875

https://twitter.com/Prof_Katz/status/476396204104757249

What I got back seemed defensive (and clueless):

Trying to be nice one last time, I responded:

https://twitter.com/Prof_Katz/status/476396566215802882

There’s a stance in his comment that I find fascinating, however.  It is one that is impervious to even considering a critique and relies heavily on ignorance to bolster a position that is not especially strong.  I have no idea if Mr. Bowdon has no working knowledge of the difference between descriptive and inferential statistics, but he displays no interest in finding out here.  Further, he retreats into calling my objections “convenient” rather than inquiring after their substance.

His next tweet:

Ah, yes, “apologists” – boilerplate anti-union talk wrapped up in a thorough lack of understanding of how research is conducted whether deliberate or otherwise.  His complaint here is instructive, however.  Saying “any study can be called ‘incomplete'” covers the fact that the NCTQ report can barely be called a “study” because it really has no research questions but it tries to make inferences from the incomplete descriptive statistics that it employs.  Calling teacher “absences” “hard objective stats” is, I suppose, some attempt to claim there is no ambiguity in the data and any inferences we want to make from them can be made by appealing to “obviousness”.  This is how someone thinks when they truly do believe that you can make “statistics say anything” but it not anything that would be said by any honest person who understands reasoning with statistics.  For example, the NCTQ data suggests that teachers are absent more than the workforce average, and from that, they infer a need for policy interventions.  But without knowing the REASONS for teachers being absent, or if such absence rates are actually significant, it is impossible to make any informed comments on existing or possible interventions.  What if teachers are absent more than average because they work in close contact with children and get sick more often than the rest of the workforce?  What if teachers are more absent because the teacher workforce is still largely female and women still bear a disproportionate share of child care duties in American families?  Policy interventions for either of these circumstances would be radically different, but Mr. Bowdon is obviously only interested in blaming teachers specifically and unions generally.

So I took out the teacher in me:

 

https://twitter.com/Prof_Katz/status/476397944908693504

https://twitter.com/Prof_Katz/status/476467588969406464

https://twitter.com/Prof_Katz/status/476467725435297792

https://twitter.com/Prof_Katz/status/476467899679244289

https://twitter.com/Prof_Katz/status/476468053916409856

https://twitter.com/Prof_Katz/status/476468184921276416

https://twitter.com/Prof_Katz/status/476468316215578624

https://twitter.com/Prof_Katz/status/476468623502868480

https://twitter.com/Prof_Katz/status/476468778050392064

https://twitter.com/Prof_Katz/status/476468970485063680

https://twitter.com/Prof_Katz/status/476469148256460800

https://twitter.com/Prof_Katz/status/476469424518463488

https://twitter.com/Prof_Katz/status/476469546040037377

https://twitter.com/Prof_Katz/status/476469926241116160

Okay, fine, that’s a lot to read, but I was surprised that he tried to as if I hadn’t just written to him like he was capable of understanding my points:

I admit that baffled me.  But I have been teaching for twenty years, and I think that even the most obstinate of students is capable of a breakthrough:

https://twitter.com/Prof_Katz/status/476479886052163585

https://twitter.com/Prof_Katz/status/476480085210722307

https://twitter.com/Prof_Katz/status/476480338957332481

https://twitter.com/Prof_Katz/status/476480398700998656

https://twitter.com/Prof_Katz/status/476480524983078912

Mr. Bowdon was still pretty oblivious:

Sometimes you have to throw in the towel:

https://twitter.com/Prof_Katz/status/476493828291981312

Mr. Bob Bowdon is not an isolated case of low information punditry trying to stake out ground as a “thought leader” in education.  He calls his website a national “hub” for online information about education reform, but his bias in that space is obvious, where he actively champions education “reformers” seeking to increase testing, spread charter schools and curtail teacher unions and where he labels people who oppose such efforts as the education “establishment”.  Mr. Bowdon finds the NCTQ report important not because it reveals interesting and important insights into the teacher workforce (it doesn’t), but because he can use it to argue that unionized teachers abuse the “perks” of their employment. This is similar to no end of billionaire-funded efforts to fundamentally change the way we offer schools without a public vote and which dare to call themselves civil rights movements.

It may be momentarily fun to demonstrate how low knowledge many foot soldiers for corporate reform are, but it is also entirely serious.

Leave a comment

Filed under Activism, Data, Media, NCTQ, schools, teaching