College of Education > News and Publications > News: Jan. - March 2011 > Making Sense of the Evidence Surrounding the Preparation of Teachers

Making Sense of the Evidence Surrounding the Preparation of Teachers

Dean Monk's column for Connections, January 2011

 

Dean Monk's column for Connections, November 2010

dean_monk.jpg

Dean's Message

(January 2011)

 

We live in a world where significant disagreements exist over how best to prepare the next generation of teachers. Because the disagreements hinge to an extent on the interpretation of available evidence, from research as well as day-to-day experience of educators and children, and because the improvement of teaching is widely understood to be crucial to the nation’s well being, Congress turned to the National Research Council (NRC).

The NRC has a long and distinguished history of providing advice on technical matters to the Federal Government. The NRC was established in 1916 as the operating arm of the National Academy of Sciences (NAS), a private organization chartered by Congress in 1863 to provide scientific advice to the federal government. The underlying principle of the NRC is that real evidence interpreted by knowledgeable and impartial individuals offers important guidance for the government’s efforts to create sound public policy. Panels of experts are appointed by the NRC on timely topics where there is a need to know more and where disagreements surround the meaning of the evidentiary base. Members of these panels have relevant expertise and experience, are known for their accomplishments in the world of research and practice, and come from diverse backgrounds. In accordance with the NAS charter, panel members, who are appointed by the president of the Academy, receive no compensation for their efforts. The goal is to take the high road and to offer the voice of informed reason in what are often contentious debates.

I was privileged to be a member of the NRC panel charged with examining the available evidence and offering advice about the current and future condition of teacher preparation in America. The panel worked hard over five years to assemble and make sense of the available evidence. Our report, available online, makes several significant recommendations, including recommendations about the kind of research needed for the future.

This was a challenging assignment in part because of the considerable variation and inconsistency across the states in the types of available evidence. But even more vexing was how difficult it was to reach consensus because of the substantial differences in initial perspectives across the committee about what should count as evidence. In this matter the panel was, indeed, representative of the broader community of scholars and policy makers among whom the debate about evidence and evidentiary standards has long been a topic of concern and contention.

Some panel members felt the only evidence worth considering are the results of experiments where treatments are randomly distributed across broad populations of students, schools, or school districts and from which the results are interpreted in terms of cause and effect. In other words, this “camp” advocates for research that mimics, to the extent possible, experiments that are conducted in the world of medicine, public health, and biological laboratories.

Other panelists were more willing to accept as useful evidence professional judgments that have evolved over decades of experience with alternative approaches and more informal assessments of consequences. For this group, both the limitations of experimental research and the benefits of experiential wisdom give credence to a more eclectic approach to the question of evidence and evidentiary standards.

The consensus that (finally) emerged recognized the strengths and limitations of different research designs and led to recommendations for more comprehensive and complementary types of research. It is clear that for some questions in education, experimental research can contribute critically important information; and in fact there has been a significant expansion in the number of experimentally based studies of education, although even now relatively few completed experimental studies exist. Thus, the “experimental design only” camp is pushed toward the conclusion that because the evidentiary base is thin little advice can be offered. Views like this fuel the broader nihilistic conclusion that we know next to nothing about how to prepare teachers and leads to an even more dismal conclusion that, since we don’t know anything “with scientific certainty,” one approach is about as good as another. If the “anything goes” view prevails, in these days of tight budgets the least expensive approach is going to win the day.

It is hard for me to square the “anything goes” view with the day-to-day realities I see in efforts to provide high-quality teacher preparation. I also wonder about what happens in experimental studies where the design includes deliberate efforts to wash out the effects of outliers. The thinking is that outliers are misleading and that it is best in experiments to calculate average effects rather than to focus on the unusual or outlying effects. In education, I wonder if we are not better advised also to pay explicit attention to unusual effects.

I like to think that everyone can remember at least one teacher who had a truly transformational impact on his or her life. But even if not everyone has had this experience, some of us have been lucky enough to encounter this kind of teacher and for me this speaks to the importance and power of outliers and unusual effects.

I received a phone call completely out of the blue recently from a former student I had when I began my career as a third-grade teacher in an inner-city school. She had been searching for me for years and simply wanted to tell me that I made an important difference in her life. It was the ultimate feel-good moment for a teacher that this woman, now in her mid-40s, wanted to say, “thank you.” I mention this not because I was such a great teacher or because I think one good story is sufficient as a basis for broad public policy. In fact, in many ways, as a first year teacher, I suspect my performance was far from excellent.

Rather, I mention it because anecdotal though the story is, it is a story that speaks to the potential for education to be transformational. We lose track of these transformational effects at our peril, and I wonder how many experimental studies we need to conduct where the finding is that the average effect is no different before we realize we may be losing sight of education’s transformational potential.

I hope the NRC report proves to be helpful in the debate over how best to prepare teachers. The fact that a diverse committee—with divergent and strongly held views about the nature of the problem and the evidence available to support solutions was able to reach consensus on its key recommendations—is worthy of attention by policy makers, educators, and the general public. We strove to provide a careful, measured, and balanced assessment of the available evidence and artfully dealt with the question of what counts as evidence. We offered explicit advice about the kind of research that needs to be conducted in the future, and I remain hopeful that this advice will be heeded.

 


David H. Monk

Dean

 

The author wishes to thank Dr. Michael Feuer, formerly the executive director of the Division of Behavioral and Social Sciences and Education at the NRC and currently dean of the College of Education at George Washington University, for background information about the NRC and helpful comments on an earlier draft.