The Politics of Knowledge
Why Research Does (or Does Not) Influence Education Policy
About This Event

Today, increasing attention is being paid to the importance and rigor of education research, which includes data collection and case studies of teaching practices, student achievement, and education policy. The No Child Left Behind Act’s call for interventions based on “scientifically based research,” the Education Sciences Reform Act of 2002, Listen to Audio


Download Audio as MP3
and a flood of data on student achievement have raised hopes that education research’s day has finally come. These rising expectations have been coupled with an influx of sophisticated research designs, yielding a growing body of research that dramatically expands what is known in the fields of teacher quality, school choice, and reading, among others. Nonetheless, there is frustration among researchers and reformers that this research too rarely influences policy or public understanding, and is too often twisted by advocates to suit their particular aims. At the same time, changes in research institutions, technology, and research funding have upended the ways in which research findings are communicated, thus offering new opportunities--but also raising concerns about how research is monitored, evaluated, and consumed.

AEI resident scholar and director of education policy studies Frederick M. Hess has commissioned eleven papers to examine how and why high-quality research influences policy, how research is used (or misused) in core policy areas, and how education research is consumed by key audiences. Please join us as AEI hosts a conference at which panelists will present their findings on education research and explore their implications for school improvement. Researchers and discussants will suggest how incentives and institutions can be altered to encourage rigorous research and its proper use, while recognizing its limits.

Agenda
8:30 a.m.
Registration
9:00
Introduction:
Frederick M. Hess, AEI
9:10
Panel I: The Evolving Relationship between Research and Policy
Presenters:
Jeffrey Henig, Columbia University Teachers College
Andrew Rudalevige, Dickinson College
Discussants:
Gina Burkhardt, Learning Point Associates
Michael Feuer, National Research Council
Ellen Condliffe Lagemann, Harvard University
10:35
Break
10:45
Panel II: How Research Is Used—Teacher Quality and Reading
Presenters:
Richard Ingersoll, University of Pennsylvania
James Kim, Harvard Graduate School of Education
Discussants:
Reid Lyon, Higher Ed Holdings and Whitney International University
Lorraine McDonnell, University of California, Santa Barbara
12:00 p.m.
Luncheon
12:45 p.m.
Panel III: How Research Is Used—NCLB and School Choice
Presenters:
Michael J. Petrilli, Thomas B. Fordham Foundation
Andrew Rotherham, Education Sector
Discussants:
David Driscoll, Massachusetts Commissioner of Education
Roberto Rodriguez, United States Senate HELP Committee
2:00
Break
2:10
Panel IV: How Research Is Used by the Public, the Courts, and Educational Leaders
Presenters:
Lance Fusarelli, North Carolina State University
Joshua Dunn, University of Colorado at Colorado Springs
Discussants:
Pascal Forgione, Austin Independent School District
William Howell, University of Chicago
Warren Simmons, Annenberg Institute for School Reform
3:30
Break
3:40
Panel V: Changing the Incentives for Researchers and Decision-Makers
Presenters:
Dan Goldhaber, University of Washington
Kenneth Wong, Brown University
Discussants:
Michael McPherson, Spencer Foundation
Kathleen McCartney, Harvard Graduate School of Education
Grover J. (Russ) Whitehurst, Institute of Education Sciences
5:10
Reception
6:00
Adjournment
Event Summary

May 2007

The Politics of Knowledge: Why Research Does (or Does Not) Influence Education Policy

Today, increasing attention is being paid to the importance and rigor of education research, which includes data collection and case studies of teaching practices, student achievement, and education policy. The No Child Left Behind Act's call for interventions based on "scientifically based research," the Education Sciences Reform Act of 2002, and a flood of data on student achievement have raised hopes that education research's day has finally come. These rising expectations have been coupled with an influx of sophisticated research designs, yielding a growing body of research that dramatically expands what is known in the fields of teacher quality, school choice, and reading, among others. Nonetheless, there is frustration among researchers and reformers that this research too rarely influences policy or public understanding, and is too often twisted by advocates to suit their particular aims. At the same time, changes in research institutions, technology, and research funding have upended the ways in which research findings are communicated, thus offering new opportunities--but also raising concerns about how research is monitored, evaluated, and consumed.

 

AEI resident scholar and director of education policy studies Frederick M. Hess commissioned ten papers to examine how and why high-quality research influences policy, how research is used (or misused) in core policy areas, and how education research is consumed by key audiences. On May 21, 2007, authors presented their findings on education research and explored their implications for school improvement. Researchers and discussants suggested how incentives and institutions could be altered to encourage rigorous research and its proper use, while recognizing its limits.

 

Frederick M. Hess

AEI

 

We have spent a great deal of time in recent years discussing the merits of various methods to enhance the rigor and reliability of research and evaluation, as well as how to communicate findings to practitioners in schools and classrooms, so that new findings will be utilized.

 

We have spent far less time thinking seriously about how efforts to promote the rigor and relevance of research are positioned within the political and policy environment--about how political pressures shape the research agenda, how research is communicated or fed into the policymaking process, or how relationships between researchers and public officials (and advocates, educators, and the media) affect the utility and the use of research. Instead, we have been more likely to hear, or to voice, broad-brush denunciations of the educational research community, think tanks, and "politicized" decision-making.

 

Our focus today is the "soft tissue" that links these two worlds and the institutions, norms, and incentives that shape their respective activity. The guiding assumption is that we can get all the technical and scientific questions right and still not benefit from scholarly findings--and, conversely, that the value of research will be dramatically heightened if these are sensibly constructed.

 

Panel I: The Evolving Relationship between Research and Policy

 

Andrew Rudalevige

Dickinson College

 

Since 1965, there has been a sequence of organizational manifestations in education research. The Office of Education was established in the 1960s, the National Institute of Education in 1972, and the Office of Educational Research and Improvement (OERI) in 1985. The Institute of Education Sciences (IES) was created in 2002 as a major reorganization. The aim of these organizations has been to isolate federal research from partisan and political influences and provide an institute that can be looked to for education research just as the National Institutes of Health (NIH) is looked to in medicine.

 

It is a perpetual struggle to achieve isolation and autonomy from politics when structures are being reorganized. Education research is hard to do in a scientific manner. It is value-laden. It is also therefore inherently political and has been politicized. In the modern era, the incentives to politicize research have at times been stronger than the incentives to isolate it. Legislative interest has been sporadic, turnover has been frequent, and research has been seen as a political arm of the administration. Can we insulate it from partisan buffering? Can we have appointments that are not political? Can we have fixed terms? Can we have a prestigious advisory board that buffers the institution from partisan politics? Can we have rigorous peer review? To make these things happen, we need time, patience, and money.

 

The ongoing debate about measurement and standards, coinciding with the political focus on accountability, has lead to the emphasis on scientifically based research in No Child Left Behind (NCLB) and the Education Sciences Reform Act of 2002. IES has provided definitions of what research entails in terms of evidence-based interventions and controlled experimentation, a fixed term for a presidentially appointed director and the National Center for Education Statistics commissioner, an advisory board, and a shift from research and development to evaluation. Some argue that we have reached a tipping point toward serious education research at the federal level. Others argue that there is too much science--narrowly defined--that does not comport with how education works.

 

Will existing incentives continue to match the needs of the research community? Will politics continue to be amenable to the kind of structure we have in place? And even if the current structure is a good one, will it have the resources it needs to succeed? Politics is about who gets what--when, where and how. Will education research get its share?

 

Jeffrey Henig

Columbia University Teachers College

 

It may be the best of times for education research. The frequent references to evidence-based research in NCLB have underscored the important role of objective knowledge in a democratic society. With grant policies, the promotion of randomized field studies, and the What Works Clearinghouse, IES has provided a picture of what strong research design looks like in education policy. Research findings and debates get coverage in publications like the Wall Street Journal, Education Week, and blogs.

 

It may also be the worst of times. Research in education often presents an ugly face. Four research battles have shown education research to be partisan, personal and bombastic: John Witte and Paul Peterson on vouchers in Milwaukee, the American Federation of Teacher's charter school study and the Center for Education Reform's ad in the New York Times, Jesse Rothstein and Caroline Hoxby on intra-metropolitan school competition, and the internal war in the U.S. Department of Education on reading.

 

Six broad structural changes are altering the demand for research, the availability and type of data, and the way that research enters the public realm. First, new technologies in the dissemination of research have altered the relationship between research and policy by increasing the speed with which research can be communicated and reducing the role of education journals. This may mean that research is cited before peer review and careful analysis, possibly eroding quality control. Second, the academy has changed as a context for education research as the relationship between disciplinary and interdisciplinary programs has evolved. Fragmentation of education research may undermine the image of academic research and its utility to policymakers, journalists and citizens.

 

Third, the growth of privatization and the increasing role of for-profit educational management organizations; the publishing industry; for-profit providers of professional development, curriculum, and testing; the new Social and Economic Science industry; and the direct delivery of for-profits and non-profits of charter schools have put new constraints on research and research institutions. Fourth, the emphasis on evidence-based research and stronger research designs has placed increased demands on a constrained budget. Of the limited funds for research, more than half is going out in the form of contracts rather than grants, with tighter oversight as a result. Researchers have turned to foundations for support, but many foundations prefer to put their money into research that supports the foundation's mission.

 

Fifth, the courts have become a consumer of research. During the 1960s, courts looked for data relevant to the cases on school desegregation. Today, there has been a shift from race to finance and from equity to adequacy, placing the emphasis on cost-benefit ratios and economic models. This may be lowering the temperature on education research by distancing it from historically heated debates, but it may also embroil education research in a new set of controversies. Finally, the sixth structural change has to do with the dynamics of federalism and the movement of education research from the local to state and federal levels. The local education bureaucracies have a tight hold on education data, which encourages researchers to align their work with the interests of the local district.

 

Ellen Condliffe Lagemann

Harvard University

 

Owing to its association with a feminized profession and its frequent association with schools at the peripheries of research universities, education research suffers from three major problems: it has not drawn sufficient resources, it has not gained high levels of respect even when it is deserved, and it has too little rigor in solving the complex problems that education presents. Education research takes place in many different places, not just in schools of education, but throughout universities, government agencies, think tanks, and foundations. People from various disciplines work in this field. Thus, education research has never developed a coherent set of standards or a clear consensus concerning central questions.

 

As long as teachers are criticized and education is seen as something that anyone can do well, research about education will not be understood as the complicated, difficult, and important undertaking that it is. We can change century-old perceptions of education research by producing excellent work and criticizing work that is not excellent. We can also do this by talking as widely as we can about education research, its importance, its demands, and its difficulty. Speaking truth to power in the public arena is vital. In addition, we need to admit that the canons of research that are likely to earn tenure or credit within the academic and scholarly community are not those that are likely to be useful. There are many disincentives to doing such work. Overcoming those disincentives will require fundamental reform of universities. Politics tends to make it very difficult to mount programs of high quality, reliable research in education. Doing so will require sustained and determined effort.

 

Gina Burkhardt

Learning Point Associates

 

To move education research forward, knowledge of politics is just as important as the politics of knowledge. There are three areas in which knowing politics helps advance a cause. The first is how research is moved to the field, how research is published in scholarly journals, and how its presentation is politically dictated. The second is how research is applied and utilized. The third is who is included in the elite research community.

 

The research community has developed a strong relationship with funders and with the Department of Education, but it has yet to figure out how to disseminate information out into the public so that it is used and applied. There has been a perpetuating myth that the Regional Education Laboratories (RELs) were generating research over the last twenty years. Until new mandates arrived from the IES, however, they were really the arm of the Department of Education charged with doing research and development. The responsibility of RELs was to take what the research community put forward and work with policymakers at the state and local level to see how the research was implemented and run pilot programs. The scale-up and its cost have been perpetual obstacles.

 

The Department of Education has not evolved well until recently with the development of the IES. The IES has changed the conversation about the rigor of research and promoted needed research. But the IES is still struggling to develop a relationship with policymakers.

 

Finally, innovation will trump the discussion of education research. Technology, business, and the media are on the brink of forcing change from the outside. There is a larger force outside of education research demanding that different kinds of education reforms be developed in response to the different needs that children have. As we struggle with education research, there is a lot of money going into schools and classrooms not based on research.

 

Michael Feuer

National Research Council

 

There is a difference between the reputation and the quality of education research, and its bad reputation is not necessarily deserved. People who write about the quality of education research need to be aware of the interpretation of research. If we believe that education research should have a bigger impact on policy, we would probably be better off studying the constraints of how education research is used rather than indicting the entire enterprise. Even if we all agreed that some education research meets the highest possible standards of quality, that is by no means a sufficient condition for the research to be understood and applied in the real world.

 

Even if research is good by objective standards, that is insufficient for its acceptance and adoption. There is a macro-level analogy to something that people who study individual decision-making have known for a long time: Rational thought does not necessarily produce rational behavior. But it is too easy to ridicule research, but the ridicule itself is not scientific. If examined systematically, a sample from the AERA catalog may show a large number of sessions dealing with evidence on the effects of high-stakes test-based accountability, the implications of brain-science for education, and the cognitive basis for science education reform.

 

Panel II: How Research Is Used--Teacher Quality and Reading

 

James Kim

Harvard Graduate School of Education

 

The "reading wars" have revolved around two concepts: the efficacy of opposing instructional practices in teaching reading and professional autonomy in terms of curricular decisions. Because these "wars" are based on both instructional and political questions, they have been fought in two different arenas. To address the instructional question, scholars have conducted research to reach consensus. To address the political question, "whole language" theorists have advocated teacher autonomy by disseminating research to professional organizations and state local boards of education.

 

Airing this debate in scholarly journals has its downsides, as it inevitably involves decades of accumulated research. Research is almost always one or two steps behind the demands of practitioners. Scholarly panels typically exclude teachers from the policymaking process. Fighting the reading wars through political forums also has its drawbacks. Whole language theorists were so effective in appealing directly to teachers that their programs were scaled up dramatically, despite a lack of supporting evidence. Because whole language has been accepted as conventional wisdom, politicians have turned to it during periods of declining reading skills, at times instituting external state mandates that undercut the professional autonomy theorists had hoped to secure.

 

The major issue in the reading wars today is not science but politics. Scholarly adversaries continue to argue back and forth about interpretations of the research. Teachers continually complain that they are excluded from the decision-making process. We need to deepen support for research among key constituents who are being asked to carry out reforms, particularly teachers. By including teachers in expert panels and by encouraging adversarial collaboration among scholars, we can improve the legitimacy of both researchers and practitioners.

 

Richard Ingersoll

University of Pennsylvania

 

Education Department data indicate that "out-of-field" teaching is a nationwide problem. These findings were alarming and captured much attention from politicians, researchers, advocacy groups, and the media. This was at first exciting, but quickly became frustrating as the data were misunderstood, misinterpreted, and misrepresented.

 

This misunderstanding always seemed to revolve around the "why" question. Two theories dominated public discourse. First, people faulted teachers, their education, preparation, and training. The data, however, showed that teachers are well-trained. This was not a question of preparation but one of fitness for a particular subject. Second, people concluded that there were teacher shortages, leading principals to settle by assigning teachers to subjects outside of their expertise. The data did show hiring difficulties, but shortages could not explain why we have high levels of misassigned teachers in surplus fields.

 

Here is a different theory: at many times, principals may find it effective, cheap, and convenient to put square pegs in round holes. They are required to coordinate a limited staff to accommodate the wide menu of courses that the public demands. In many cases, a principal's decision to misassign teachers saves money for schools and taxpayers. But these decisions are not cost-free, and with NCLB, they are now illegal.

 

Everyone believes there is a teacher deficit, either in terms of qualification or quantity. Either there are too few teachers or teachers are poorly prepared. But the data show that if we increase the number or quality of teachers, it will not stop them from being misassigned. The problem is in how schools are managed.

 

Lorraine McDonnell

University of California, Santa Barbara

 

Education researchers use traditional social science methods. Researchers rarely subject their findings to the kind of financial, political, and administrative feasibility analyses expected in policy research. Policymakers seek research that is communicated in an understandable way, even if it relies more on untested theories than validated findings.

 

The effective theorist understands the importance of "policy entrepreneurs"--the people skilled in moving ideas through a fragmented institutional system with multiple points of access. Research findings may be modified upon factoring in real-world constraints and the limited supply of policy entrepreneurs. There may be a tradeoff for researchers: while the policy entrepreneurs are essential in this process, they may adopt research and then misunderstand or misapply it.

 

Policymakers are most interested in research on issues over which they have measurable influence, and they recognize that the leverage they have over individual classrooms and schools is very limited. Policymakers will misapply research to fit requirements they can easily control, like teacher training or licensure. Thus, research pointing to changes in well-established school or classroom practices may require a more circuitous route from research findings to policy or require more supporting research.

 

Ideas like teacher professionalism and whole language become the policy images that define how particular policies are discussed and understood. They combine (sometimes misunderstood) information with ideological and emotive appeals. Ideas are critical to policy practice, particularly as frames of reference that buttress or challenge existing policy. But many education researchers are uncomfortable thinking about applications of their work as ideas.

 

Reid Lyon

Higher Ed Holdings and Whitney International University

 

Why is it difficult for trustworthy science research to inform education practice? In developing the legislation for Reading First and NCLB, it was impossible to get traction until we started to talk about instruction.

 

With Reading First, we have had trouble infusing what we know about how children learn to read into the fabric of schools and curricula. The anti-scientific culture is probably the fault of the middlemen who do not present research in a relevant or readable way. This culture is one where practices are driven by tradition, philosophy, and untested beliefs--concepts that resonate quickly. In the experience of Reading First, there was a conversation between researchers and a few key policymakers. We set clear definitions and criteria and we emphasized that Reading First was for specific situations under certain conditions.

 

Our researchers continue to polarize the education community. The debates are not about the quality of research we have. They are about training those who use research to apply appropriate methods. We must avoid a one-size-fits-all philosophy of research methods.

 

Panel III: How Research Is Used--NCLB and School Choice

 

Michael J. Petrilli

Thomas B. Fordham Foundation

 

There are two key elements of the relationship of NCLB to education research. First, there is the idea on Capitol Hill that NCLB's programs should be funded based on scientific research. Second, there is the idea of whether NCLB itself can be said to be scientific and whether research has informed the NCLB debate.

 

In the text of NCLB, there are 216 references to research, and 54 percent of the time, it specifically mentions scientifically-based research. It seems that Congress decided that some things should be based on scientific research and other things could be simply based on research of a lower level--arts programs, for example. But it appears that Democrats suggested programs and Republicans refused to implement them without language requiring a basis of scientifically-based research. On both sides, some admitted to putting in language in areas of the law where there was no research. They justified this action, saying that the language created a demand for research in some areas that did not have it. There is some evidence that this has indeed changed the conversation about education and research.

 

For the latter question, to determine whether NCLB itself was scientifically based, we can look at the hearing process from 1995 to 2001. The legislative history of this period showed witness testimony from local level officials, advocacy groups and researchers. Researchers tended to have smaller hearings, testifying 17 percent of the time, but they were very much proactive in the hearing process. Research added some substantive value to NCLB policymaking, as shown through concern on the Hill for studies on teacher quality. Members of Congress and their staffs, however, tended to gravitate toward ideologically appealing studies.

 

Researchers can get their work noticed by seeking out synthesizers. Alternatively, they can circumvent these groups altogether by reaching out to policymakers directly through op-eds, policy briefs, and the like. Policymakers need to be aware that people have various agendas and they should ask researchers and synthesizers to work on a simple question and give them the whole story.

 

Andrew Rotherham

Education Sector

 

The study of school choice research and the extent of its impact is a challenging assignment because school choice is a very ideological concept. Evidence changes few minds on either side of the debate regardless of its quality. The issue can be broken down into two questions that are frequently conflated. First, what range of educational choices should government finance? Second, what are the effects of the decisions policymakers can make about educational choice?

 

The first question is fundamentally ideological and competing views are thus legitimate--research should not be used as a sort of proxy for these views. The second question can be looked at through the lens of three separate issues that surround the issue of choice: politics, research debates, and the media. The politics behind this issue are ideologically driven. The politics are organizational in the sense that large organizations have vested interests in the current system. The framing of the issue often matters more than evidence.

 

The research on school choice has moved from being descriptive to inferential. The early work was theoretical partly because there was no data. Recently, the research is more inferential partly because of the research standards movement and availability of data. In the larger scheme of the debate, relatively small issues tend to prevail on either side. There are some key challenges in the research, such as data availability, peer review, and the incentives for researchers.

 

One of the striking features of the debate is just how personal it can become in the media. Journalism and social science are often at odds with each other. Both fields prize accuracy, but there are some key differences. Journalism is fast-paced and time-limited; social science is built on the steady accretion of knowledge. Some education studies are very complicated, and the answer is often, "it depends." But many journalists want immediate answers.

 

Ideology, not research, is shaping the school choice debate. While the research can inform discrete aspects of the choice question, it cannot simply answer the question because of dynamics surrounding the role of the government, the level of investment, the level of state participation and public opinion.

 

The Honorable David Driscoll

Massachusetts Commissioner of Education

 

Many instances of inconsistencies in education performance and research evidence have shown over the years that the exact science of ideological politics always trumps the inexact science of research. For example, charter schools consistently come out with reports that show the charter schools are working. The opponents of charter schools report that they are not working. Both groups use the same set of statistics.

 

The responsibility therefore rests on local practitioners to understand and apply research. Education research in itself should serve as a motivator for education stakeholders to stand up for what they should believe in. For example, when we talk about letting the media act as translators between the education policy arena and the public, or when we talk about the back and forth between Republicans and Democrats on scientific research language in NCLB, we must control the situation and make sure the right things are said and done, for the right reasons.

 

It is especially important for practitioners to discern what has worked well in the past and think about what will work in the future, based on this preexisting knowledge of the research. Practitioners must have a good knowledge of preexistent and proven frameworks to assess the relevance and practicability of education research in the field of education administration.

 

Roberto Rodriguez

United States Senate HELP Committee

 

The legislative process of NCLB is dynamic and fluid but not perfect. The research is good, but policymakers should do all they can to listen to research and struggle with the question of how to better link research to policy and practice in legislation. The move to back up legislation with scientific research was not wholly new to the practices of federal education policy, but the focus has shifted to it.

 

To move forward on NCLB, we must first note that the act does a good job with assessment, accountability, and standards. Its basic architecture is sound. A number of states have now identified schools in need of improvement, so the new question is an old one: how to make and sustain change in our lowest-performing schools? This is a point to which research lends itself well.

 

But we must proceed with caution in using education research. We must consider issues of research quality, methodology, ideological agendas, and research conflicting with compelling public interest. On the first, we retain standards of research quality. Research should not cross the line of an explicit financial or political interest. Finally, in some areas we need to step back and ask what the compelling public interest is for a particular policy and not necessarily let research get in our way.

 

Panel IV: How Research Is Used by the Public, the Courts, and Educational Leaders

 

Lance Fusarelli

North Carolina State University

 

Thanks to evolving state accountability mandates and NCLB, there is increasing evidence that data-based decision-making is taking place in school administrations. It is still unclear, however, why educational leaders are not learning directly and effectively from the "experts" in education research. Structural and personal barriers impede a tangible alliance between education research and its use in policy practice. For example, current trends show that school leaders use research unevenly and that school practice reforms are usually externally imposed.

 

School officials are faced with the daunting task of navigating the many conflicting findings. The academic community generates limited prescriptive research, leading to skepticism about its usefulness, as does the surfeit of poor-quality education research. Practitioners lack time or expertise to navigate the research as they often find themselves in a constant crisis-management mode that makes it difficult to engage in systemic strategic planning. Practitioners also have a general preference for local context and experience in discussing education reform.

 

NCLB, which encourages more data-based decision-making and training programs, has created a pathway to progress. The role of IES in funding data academies, which promote data-based research use, is also important. We need more localized research consortiums in large urban systems and more research in practitioner journals. These steps should help resolve the gap between the creation and use of education research.

 

Joshua Dunn

University of Colorado at Colorado Springs

 

At the intersection of education policy and politics is a great deal of judicial activity. Concerning court-specific trends, the application of social science research has been inconsistent. For example, the Supreme Court has no obligation to consider research-based evidence. Also, while the courts guard against the infiltration of "junk science" through the Daubert Standard, education research does not fit neatly into these standards because there is no established metric for what "good" material is. Finally, education litigation has moved from federal to state courts. State courts are more political in their methods of data selection and retention, which actually makes them more likely to rely on social science testimony.

 

The most significant use of the research has been in the areas of desegregation. We can still question the legitimacy of using social science research to determine the meaning of constitutional clauses. After Milliken v. Bradley created a doctrine for compensatory remedies and made educational experts essential to the remedy design process, the quality of research in this area has come under scrutiny. In the later stages of desegregation proceedings, however, education research has declined in importance.

 

Meanwhile, the majority of the research used by the courts continues to be in school finance. After Rodriguez v. San Antonio, the issue turned to the adequacy of school funding. Education research was essential in this area to determine whether state constitutional mandates were being upheld.

 

There continue to be concerns about the utility, legitimacy, and quality of education research. Frivolous usage of research in the courts only makes the situation worse. Additionally, the politicized relationship between research and advocacy has undermined people's faith in the research.

 

Pascal Forgione

Austin Independent School District

 

It remains difficult to establish a positive relationship between research and practice. While one might think that research can be easily applied to school decision making, the truth is that schools and school districts rarely use all available information.

 

High-performing school districts such as Austin's can provide insights on streamlining the school decision-making process with cohesive research use. Such districts are more productive because they make decisions in a systematic manner using research in district-wide teaching and learning improvements and in creating a common language to facilitate this guidance.

 

No magic bullet strategy can bring education practices up to par--we must have standards. Research standards provide objectivity, raise critical analysis and thinking to a deeper level, and raise tough questions that do not otherwise appear. The research itself also requires an upheaval, particularly in the area of research on implementation. The IES should contribute more research on how programs are implemented in different types of school environments.

 

William Howell

University of Chicago

 

To influence policy practice, it is in academics' interest to write for a largely technical audience rather than for education policymakers. Does education research actually influence the decisions rendered in the courts? There is reason to doubt this, because research of this sort arguably does not serve as a binding constraint. It is doubtful that judges are actually ruling one way over another because of the existence of a body of research. The single best predictor of court ruling is still the court's predominant court ideology. It remains unclear in what ways and to what extent education research is an influence in the courts.

 

Keeping the above discrepancies in mind, it is important to note that academic research can discipline the conversation of education policy, particularly in setting boundaries for what constitutes a reasonable argument among reasonable people. The term "reasonable" serves as a caveat to warn us of using research irresponsibly.

 

The public has a role in navigating the education research environment because ordinary citizens are for the most part pragmatic. Policy debates will be more easily resolved with the participation of a public that willingly considers both sides of a given issue.

 

Warren Simmons

Annenberg Institute for School Reform

 

Politics is an inherent part of policymaking and research. The more pressing concern is how to design research that informs or generates policy that produces practices and outcomes of scale. A key part of the problem in forming an alliance between education practice and research is the very nature of the research. Education research is not cohesive. Only in the last decade or so have school systems been asked to create outcomes of scale. But how can a coherent alignment in the system be created on the basis of a fragmented body of education research?

 

To resolve this question, we must moderate and control research rather than insulate ourselves from the fact that the research enterprise is necessarily driven by political and ideological values. By facing the issue head on and working within it, a healthy intellectual environment of "adversarial collaboration" can generate useful education research.

 

The current policy environment promises some improvements of scale of this type, such as through school choice, school restructuring, and culturally responsive pedagogy. To make good on these promises, we must develop a set of criteria for education research. First, research quality, rigor, and reliability should be addressed. Second, the research should be relevant to questions that practitioners bring to the table. Third, decision-makers and policymakers should have some degree of ownership over research-based education methodology. And finally, districts should pay attention to community engagement, communication, and cross-sector leadership development.

 

Panel V: Changing the Incentives for Researchers and Decision-Makers

 

Dan Goldhaber

University of Washington

 

More research is generated today than ever before, but the evidentiary basis for policy questions is still too thin and research is often misapplied. Most research in education is done at universities in schools of education. Yet most researchers at schools of education do not have the methodological training to address the issues that surround experimental research. Outside of education schools, researchers generally have better training but lack the understanding of the cultures and institutions within education. Furthermore, most of the incentives that drive university researchers do not encourage research helpful to policymakers and practitioners. Rather, research is done primarily to earn tenure. That means that research is not primarily being produced for those who would use the research.

 

Research that attracts publicity competes with high-quality research. Research needs to be timely, and academics worry that other researchers will preempt their research and therefore make fewer quality checks. The What Works Clearinghouse and the new design of the research review panels at IES are good start for quality control, but we can do more. On the front end, funders--mainly foundations--can identify standards for evidence and review processes. On the back end, a flagship education journal would be a mechanism for quality control. A national test would also be helpful.

 

Kenneth Wong

Brown University

 

In order for researchers to communicate with policymakers effectively, to get them to think in terms of framing the policy debate, we need to pay attention to what fuels policy decisions. Politicians are concerned primarily with being reelected. Politicians are also responsible for representing their constituencies. There is a trade-off for politicians between individual self-interest and the mission to make a difference. Sometimes these incentives work together and sometimes they do not. Political leaders do not behave the same at all levels of government. Federal elected officials have a broader tax base and a broader constituency; locally elected officials have to deal with the reality that people can vote with their feet and leave the community.

 

There are 14,000 school systems in the United States. A large literature suggests that states and localities are sources of policy experimentation. Some policies in NCLB were implemented on a local or state level first; successful local programs have a tendency to be federalized. Yet research is only one of many conditions for their incorporation into national policy. Timing is everything--elections come very two to four years, while good research can often take ten years or more. Furthermore, what works in one state or district may not work in another.

 

We need to enhance the rigor and credibility of research. The National Institutes of Health (NIH) provides a useful model. NIH is comprised of statutory categorical institutes that identify strategic agendas.

 

Grover J. (Russ) Whitehurst

Institute for Education Sciences

 

While NIH is often referenced as a model for education research, it is important to realize that OERI was organized around five categorical institutes, but that this did not work well.

 

The research and policy communities are both chaotic, but there needs to a market for education research and a market for education performance. Before there is a market, there must be expectations and there must be some form of accountability for meeting those expectations. To create this market, reliable information is necessary. Once information is available, a market for education research will begin bridge the gap between research and policy community.

 

IES has a role to play in all of this: to build a strong federal research agency that can fund rigorous and relevant research. We must build capacity by increasing funding for education research and by training the next generation of education researchers.

 

Personal relationships with individuals in the right place at the right time can influence the use of education research. Continuity is also key. The federal government's funding of education agencies has been inconstant. We cannot expect people to make careers in education research unless they can anticipate that there will be a stable federal structure to support that research.

 

Michael McPherson

Spencer Foundation

 

We need to address the question of how education research actually influences education research. Before we fix anything, it is important to figure out how it works. There are two reasons that this question does not get studied more. First, we are asking extremely complex causal questions. People hold complex sets of knowledge and belief structures and it is necessary to determine how systematic social science research factors into how individuals make decisions. Second, social scientists approach these questions with preconceived notions of the function of causal research and frustration with how the research is used.

 

The implicit model for policymakers to frame clear alternatives and seek evidence about which alternatives will produce the desired results, and we think the way to do so is to conduct expensive random control experiments. The influence of social science is often much more indirect. Rather than answering the questions policymakers ask, social science changes the questions and causes policymakers to frame the questions differently.

 

There is too much interest in streamlining the process between what we perceive as social needs and what the academy works on. Ever since the Middle Ages, academic work has served to feed the existing structures, but it has also served the critical role of asking questions that are not being asked elsewhere and questioning the existing system. What researchers study and what they care about is determined by their own academic values. We need people who examine and challenge the questions that policymakers pose.

 

Kathleen McCartney

Harvard Graduate School of Education

 

Education scholars need to shift from seeing themselves as expert problem solvers to seeing themselves as participants in democratic deliberations. Decisions should reflect the will of the people, and our goal should be to ensure that our data are part of the discussion.

 

Two obstacles prevent researchers from influencing policy. The first is that data are imperfect. It is rare to have definitive data on any scientific topic. When there is finally clarity surrounding educational issues, practitioners and policymakers will take notice. The second obstacle is related to the academy and its incentives for research. Ineffective programs are often implemented while effective programs are not, simply because word does not get out to practitioners and researchers move on to other projects.

 

We need fair rules for how data are used and produced. Education needs more experimental research, but not at the expense of other methods. Education also needs large-scale studies built by collaborative teams. There are a number of mechanisms that can help education schools provide incentives for research that is relevant to policy decisions.

 

We need to reject the "arts and sciences" model where success is defined by research productivity and accept our role as professional schools in which success should be defined by the impact of our research on policy and practice. Faculty members should not all be expected to do the same thing, but criteria for tenure could include impacts on practice and policy, not just publications. Testifying before Congress or to state legislatures should be just as important.

 

AEI research assistants Rosemary Kendrick and Juliet Squire and AEI intern Arushi Sharma prepared this summary.

View complete summary.
Also Visit
AEIdeas Blog The American Magazine

What's new on AEI

AEI Election Watch 2014: What will happen and why it matters
image A nation divided by marriage
image Teaching reform
image Socialist party pushing $20 minimum wage defends $13-an-hour job listing
AEI Participants

 

Frederick M.
Hess
AEI on Facebook