Do master’s degrees matter?

In the last few months, I happened to have many exchanges on this particular question. Does it make sense to require teachers to continue their professional development in a formal academic setting? Do master’s degrees or their equivalents matter? All these talks made me realize how differently people view research depending on how they perceive and understand it. In any public debate it is important to assume that your opponent is neither evil or stupid, and that differences may arise from different, unspoken assumptions, which in turn, created by different life experiences.

Those of us who struggled through a doctoral dissertation or another research project, walk away with a deep but ambivalent feeling. On one hand, we know just how messy the research process is, and how deep is our collective ignorance about the world. On the other hand, we learn to tell the few things we really do know from the mountains of common sense, nonsense, and political hype. Having a doctoral title does not make anyone smarter, but it does add both a specialized expertise and a particular ethos of looking at research claims.

For example, here is the summary of current findings, which I borrow from Dan Goldhaber and Michael Hansen’s recent article:

First, teacher quality, measured by value-added models (VAMs), is the most important school-based factor when it comes to improving student achievement. For example, Rivkin et al. (2005) and Rockoff (2004) estimate that a one standard deviation increase in teacher quality raises student achievement in reading and math by about 10 percent of a standard deviation – an achievement effect that is on the same order of magnitude as lowering class size by 10 to 13 students (Rivkin et al. 2005). Second, teacher quality appears to be a highly variable commodity. Studies typically find that less than 10 percent of the variation in teacher effectiveness can be attributed to readily observable credentials like degree and experience levels (e.g. Aaronson et al. 2007; Goldhaber et al. 1999; Third, while the evidentiary base is thin, it appears that only a strikingly small percentage of teachers are ever dismissed (or non-renewed) as a consequence of documented poor performance.

I am not going to dispute any of these. But when I read it, I think immediately: “the most important school-based factor” is teaching, yes, but the school itself is probably 15-20% of influence over student outcomes. The rest of it is the social circumstances, so why are we focusing so much attention and money on the 15%, while almost ignoring the rest: early childhood and expanded learning programming, and taking care of children’s nutrition, health, and young parent education? Even if a miracle happens and we put an outstanding teacher in every troubling school, it is not going to solve the problem.

I also reflect on how pathetically little we know about the reasons the good teachers are good. The definition of a good teacher Rivkin uses (I believe it is actually Sanders and Rivers definition from their 1996 paper) is tautological. You’re good teacher when your students progress well, this is how we know you’re good. This is more of a mystery than a finding useable in public policy decision-making. We don’t know how to predict who is going to be good, nor do we know how to improve people’s performance. We don’t even know if their success is owed to some intrinsic features of their mind and character, or to their training, or to the kind of work environment and support they receive when they start. It is very likely that a combination of all three is at work, but no one could back this up with a large-scale study. And finally, on the point that academic credentials do not predict teacher performance. I also happen to know that none of these studies so far disaggregated between degrees in education and all others. Of course, if you a teacher of Chemistry and are doing a degree in fashion design, it is likely to detract you from doing your job. But if you are an elementary classroom teacher, and you are in, say an M.Ed. in Reading or TESL, we can expect impact on student learning – no one just got around measuring them.

And that’s the problem – most people do not realize how little research is done in education. Same scholars who are now so excited about the teacher factor (economists, mostly), were commenting for decades that teaching does not seem to matter at all. It does not mean the quality of teaching was unimportant for 30 years, between 1966 (The Coleman Report) and 1996 (Sanders and Riversmpaper). The reality did not change, but our thinking has changed somewhat, and will change again. Just because no one has done a good study on measuring teachers’ academic credentials on their performance, does not mean it does not exist. It is also frustrating that much of great research that is outside of the simple linking factors to student test scores, gets no public recognition, and is routinely ignored by policymakers (just browse through recent papers of P. Grossman, D. Boyd, M. Cochran-Smith, L. Darling-Hammond, B. Floden, and many others).

But someone else looking at the same paragraph, has a very different reaction. Of course, they think, we need to find out how much student growth each teacher is responsible for, and get rid of bad teachers. Of course, they say, we need to stop requiring master’s degree of practicing teachers, because they do not matter. A similar inconclusive set of research findings exist about teacher certification, so let’s get rid of teacher training while we are at it, and replace it with short alternative training programs.

Research is a funny thing, it is double-edged. It exists to correct our common sense assumptions. However, in unskilled hands, tidbits of good research can be used to make huge policy mistakes. With the research information becoming readily available to anyone, many smart, well-intended, but unprepared people are trying to interpret it. It happens not just in education – we have millions of amateurs reading medical research, and forming passionate if not militant interest groups. Thank god, very rarely do they make a visible impact on policy.

Within the educational research community, most people are trying to behave honorably, and always disclose the limits of what we know. But their caveats and disclaimers don’t make it into the media, and ignored by overeager reformers at all levels. For example, Dine Ravitch spelled out in May of 2010 that none of the key provisions of the Race to the Top program can be backed up by research. To my knowledge, no one has disputed her claims. And yet thousands of people around this great country believe as if they are acting on a program of reforms backed by research.

We must work on reform, but only when we know our plans are going to make things better, not worse. How do we know? Research is the best option; it often cannot answer questions we are asking. The next best thing is professional consensus: it is not always reliable and subject to professional biases, but at least it provides some ways of sorting nonsense from good ideas. But we must rule out one way of reforming our education; it is when a few passionate, smart, but unprepared people misinterpret research findings and convince themselves that they know all the answers. Many of them tend to believe that things cannot get worse. In that, they are sadly mistaken.