American Enterprise Institute

November 30, 2006

[Edited transcript from audio tapes]

8:15 a.m.	Registration and Breakfast
9:00	Introduction:	Frederick M. Hess, AEI
9:10		Panel I: The Big Picture—National Implementation and Capacity
	Presenters:	Michael Casserly, Council of the Great City Schools
		Jeffrey R. Henig, Columbia University Teachers College
		Paul Manna, College of William and Mary
		Michael J. Petrilli, Thomas B. Fordham Foundation
	Discussant:	Chester E. Finn Jr., Thomas B. Fordham Foundation
10:40	Break
10:45		Panel II: The NCLB Remedies in the States
	Presenters:	Julian Betts, University of California, San Diego
		Patrick McGuinn, Drew University
		Alex Medler, Colorado Children’s Campaign
	Discussant:	John Winn, Florida Commissioner of Education
12:05 p.m.	Luncheon
12:55		Panel III: The NCLB Remedies in the Districts
	Presenters:	Stephen K. Clements, University of Kentucky
		Jay P. Greene, University of Arkansas
		Jane Hannaway, Urban Institute
		David Plank, Michigan State University
	Discussant:	Stephen Jones, Norfolk (Virginia) Public Schools
2:30	Break
2:35		Panel IV: Reconstituting Districts and Schools
	Presenters:	Bryan C. Hassel, Public Impact
		Joe Williams, Education Sector
	Discussants:	Alan Bersin, California Secretary of Education
		Morgan Brown, U.S. Department of Education
3:50	Break
3:55		Panel V: Lessons Learned
	Discussants:	Kati Haycock, The Education Trust
		Diane Ravitch, Brookings Institution
		Marshall (Mike) Smith, William and Flora Hewlett Foundation
5:00		Adjournment and Reception

Proceedings:

Panel I: The Big Picture—National Implementation and Capacity

Frederick M. Hess:  Hi, I’m Rick Hess, Director of Education Policy Studies at the American Enterprise Institute.  I would like to welcome all of you here today to join us for the conference “Fixing Failing Schools: Is the NCLB Toolkit Working?”  AEI is sponsoring this conference in conjunction with the Fordham Foundation.  I would like to thank my colleague in this, Checker Finn for being Checker. 

Many of you in this room were here when we held a similar effort about three years ago in a volume that was published under the title Leaving No Child Behind by Palgrave McMillan in 2004, in which Checker and I had the idea of looking at how the remedy provisions of No Child Left Behind were actually working on the ground.  At that point in time in 2004, these provisions were quite new.  Many people were still trying to figure out what they actually entailed and what they could look like and in fact in most states, we had only seen the public choice and supplemental service provisions begin to be implemented.  At the time, Checker and I, as we are wont to, were hopelessly optimistic.  And we also thought that it would be useful to come back and repeat that effort once these plans had more of an opportunity to take effect, to be implemented, to unfurl.  That is what we are gathered for here today. 

Now, No Child Left Behind has been a source of much discussion, analysis, and heated rhetoric in the past five years, of course.  It is no surprise; it is a big law with grand ambitions and a lot of moving parts.  NCLB addresses just about everything we care about in education – scientific research, reading, teacher quality, a host of issues.  But it is hard, of course, at least at the federal level—pioneering a testing and accountability framework. 

Now today, we are not trying to offer a comprehensive assessment of No Child Left Behind.  That is not what the authors were asked to do and that is not what any of them attempted to do.  Instead, we are looking at one half of the testing and accountability framework, the often taken for granted remedy half, and trying to make sense of how it is working and how it might work more effectively.  For those of you who want a broader brush-up on the law, Mike Petrilli and I did a little volume earlier in the year, which you will find available out in the lobby called “No Child Left Behind:  A Primer.”  I’m happy to point you to it. 

There is also obviously a lot of other good work on the law more broadly.  That is not what we are addressing today.  So it would be unfair to challenge the authors on questions which do not actually relate to what they were asked to study or write about.  The testing side of the testing and accountability equation has received extensive attention.  State officials, federal officials, a lot of academics have spent the past five years arguing and examining cut scores, proficiency targets, N-sizes, value-added metrics and related concerns. 

Now, receiving far less attention have been the accountability issues posed by No Child Left Behind.  What do we do when testing suggests that some schools are failing to make adequate progress or that some districts are failing to do so?  This is a question that has bedeviled reformers for decades.  It is not a new question.  The response adopted in No Child Left Behind was to create a cascade of remedies that would give students in these schools new options while addressing the shortcomings of failing schools. 

There are really at least three strands of thought embedded in the strategies of the No Child Left behind remedies.  One was to create new opportunities to serve children stuck in troubled schools.  A second was to pressure low-performing schools and districts to improve their performance.  And the third was to provide guidance and technical support to schools which were deemed to need that kind of assistance to improve.  In asking is the NCLB toolkit working, then, we find ourselves asking how well it is accomplishing these three sets of tasks. 

The challenge is that we do not have good or clean data on any of these questions.  In fact, the lassitude with which local state and federal officials have approached the task of collecting data on NCLB interventions is remarkable, especially in an era marked by ceaseless talk of research and data.  As you will hear throughout the day, simply locating descriptive data on the remedy provisions is no easy task and trying to assess the impact of any of them has barely been contemplated. 

What exactly are the NCLB remedies?  Now, many of you in this room are familiar with these and familiar in great detail.  Some of you do this every day -- Morgan.  On the other hand, I realize everybody in this room does not necessarily have a whole lot of time to delve through the particulars of NCLB, so it might be useful for us briefly just to run through the issues at play.  NCLB cascades are to be implemented in a series of steps once schools are identified as having failed to make adequate yearly progress. 

Ballpark, about one quarter of the schools in the nation this year are deemed not to be making AYP and about one in eight have been identified as “needs improvement.”  Once schools have entered “needs improvement” status year one, they are supposed to get technical assistance from the local education agency and they are required to offer public school choice to a particular set of students in those schools.  In year two of the remedies, they are required to make available supplemental educational services.  It is useful to think of it really as after-school tutoring, in most cases, to qualified students. 

In year three, they are required to create a corrective action plan that complies with a menu of available options.  In year four, they are required to begin the process of restructuring, and in year five they are required to implement restructuring.  If after restructuring in year five a school continues to fail to make adequate yearly progress, there is an open question as to exactly what happens.  This will be one of the topics we will speak about today. 

Now, today’s intrepid authors have set out to provide a deep and scholarly examination of how these NCLB remedy provisions are actually working.  The authors were not asked to evaluate these provisions.  In the language of the education profession, what we are looking at are formative rather than summative assessments of the NCLB remedies.  The authors have sought to offer a careful nuanced look at how the NCLB remedies are playing out in schools, districts, states and in the US Department of Education.  This research is instructive not only for deliberations regarding NCLB but because NCLB remedies include popular high-profile reform ideas such as public school choice, school restructuring and charter schooling that are actually relevant to broader debates about school improvement. 

Finally, a popular current line is that NCLB is well-designed but in implementation has sometimes stumbled.  That analysis deserves careful scrutiny and should very much be on the table as we look at these questions today.  As Checker and I wrote back in 2004, operationalizing any statute as complex as No Child Left Behind brings inevitable headaches.  Different agencies and levels of government must learn to work in new ways.  Officials must take on unfamiliar roles and educators must author ingrained routines.  As these arrangements are negotiated, a certain amount of confusion is to be expected.  Such problems are normal, usually diminished with time and experience, and are mainly of interest to students of government process. 

However, some laws also summon more fundamental woes by perverse incentives, incompatible interests or unworkable expectations.  These do not go away with aspirin and a night’s rest.  They may in fact require surgery.  Identifying successes and challenges in various locales reflecting on which problems may be alleviated with time and which require more fundamental action - this is our charge today. 

Today will unfold in five panels.  The first, already seated up here, is going to look at the big picture, offer a broad, descriptive look at what is taking place in the states and major urban districts as well as with supplemental services and with Department of Education implementation. 

The next panel will look at the NCLB remedies at the state level.  The third panel will look at them at the district level.  The fourth is going to look particularly at the questions of restructuring, both in cases of districts and of schools, and the final panel is going to see if we can synthesize what we might take away from everything that is presented today. 

Hard copies of the conference papers are available in the lobby where you entered.  They will also be available on the AEI website today, and the AEI website link will be up there throughout the day.  The revised collected analyses will be available next year in a volume that is going to be published by AEI Press next summer or fall.  But in the interim, the papers will be available in their entirety on the AEI website, so feel free to help yourself. 

Before we get started, I just want to make two requests.  One, please keep the day moving along.  We have limited between-panel breaks to about five minutes.  I ask that you please assist us in getting sessions started in a timely fashion.  Second, when asking questions from the audience, please wait for the microphone, identify yourself by name and affiliation, and actually ask a question.  We have this proud DC tradition of long orations with the question mark appended. 

Anyway, with that we are going to go ahead and get started with the first panel.  The first panel is going to offer us a broad look at how these issues are playing out nationally.  Four authors will be presenting.  We will be going in alphabetical order.  First up, writing about how the remedies have been playing out in major urban districts, is Mike Casserly, Executive Director of the Council of Great City Schools.  Prior to becoming Executive Director, Mike served as the organization’s Director of Legislation and Research for 15 years.  Most of you know Mike.  He is probably one of the most influential voices on urban schooling in the country today.  So we appreciate him taking the time and actually going out and collecting these data which is a nightmare, to put it simply. 

Speaking second will be Jeff Henig.  Jeff is a professor of political science at Teacher’s College, Columbia University.  His books include the Color of School Reform: Race, Politics and the Challenge of Urban Education and Building Civic Capacity: The Politics of Reforming Urban Schools.  Jeff’s work is focused on the boundary between private and public action and tackling social problems.  His piece today looks at the political economy of supplemental education services.  Again, not attempting to evaluate this in any “what is the impact on learning outcomes” sense, but trying to help us understand: what does this whole new sector look like that we have created under the legislation? 

Third is Paul Manna, an assistant professor in the Department of Government and a faculty affiliate with the Thomas Jefferson Program in Public Policy at the College of William and Mary.  Paul is the author of the recent book School’s In: Federalism and the National Education Agenda, which is the first serious treatment, actually, of how NCLB has been implemented in the states and considering questions of state capacity.  Paul is going to be talking about the challenges in implementation of the NCLB remedies at the state level. 

Finally, it will be Mike Petrilli, Vice-President for National Programs and Policy at the Thomas Fordham Foundation.  Mike served in the Department of Education.  Under the Bush administration, he was Nina Rees’ deputy at OII.  He helped coordinate and, in fact, helped launch some of the NCLB public choice provisions and supplemental service provisions.  He is also, as I mentioned, the author of No Child Left Behind: A Primer and an editor with “Education Next.” 

And finally, serving as discussant on the first panel will be Checker Finn.  Checker, President of the Thomas Fordham Foundation and the Thomas B. Fordham Institute, a Senior Fellow at Stanford’s Hoover Institution, Senior Editor of “Education Next.”  Previously, Checker has served as former Assistant Secretary for Research and Improvement at the US Department of Education and was a founding partner and senior scholar with the Edison Project. 

The way the first panel will work is each of the authors is going to take about 10 minutes to just present the key take-aways.  We will then have an extended period for conversation about what we might take away from this initial broad look.  Mike, would you please start us off?

Michael Casserly:  Thank you very much, Rick.  Can everybody hear me okay?  Thank you.  I’m Michael Casserly.  I’m the Executive Director of the Council of the Great City Schools, what you may know as a coalition of the nation’s largest urban public school systems across the country and an organization that actually supports No Child Left Behind, although after scanning some of the papers I’m starting to wonder why. 

I’m going to keep this very brief and give you just the highlight of the status of No Child Left Behind in the nation’s major urban school district.  We have considerably more data in my paper than I can possibly present here in 10 minutes so I’m just going to give you some of the briefest highlights.  The data that we are presenting and releasing today – actually, for the first time - come from a new survey that the Council of the Great City Schools did of its 66-member urban school districts, 36 of which have responded to date. 

There is more data coming so over the next several weeks these various figures will change somewhat.  These 36 responding districts have enrollments that are 65 percent-free in reduced-price lunch, 18 percent English-language learners, 13 percent students with disabilities, and about 80 percent students of color.  In addition, these 36 districts have approximately 7,500 schools enrolling approximately 5.1 million students. 

About 5,900 of these schools or about 80 percent of all of these schools are Title-One schools.  The number of these schools that are in NCLB’s sanction, that is, the actual sanction process, not the warning status, has grown steadily since 2002-2003 despite significant gains in the cities in reading and math on state tests and on the National Assessment of Educational Progress.  The number of schools in school improvement one, two, corrective action and restructuring increased from 975 in 2003-2003 to approximately 1,500 in 2003-2004 to 2,045 in 2004-2005 and 2,200 in 2005-2006.  Conversely, another 531 schools made enough progress in 2005-2006 to get out of sanctions or to have their sanctions put on hold.  About 30 percent of all the schools in these 36 cities are in sanction status.  This constitutes about 26 percent of all sanctioned schools in the country. 

Finally, there appears to be no particular relationship between the number of schools in sanction under NCLB and their performance on the National Assessment of Educational Progress.  Boston and San Diego for instance have similar reading and math scores on NAEP, but San Diego has 15 percent of its schools in sanction while Boston as 47 percent.  That is one of the reasons why our organization last week called for the institution of national standards.  The data also indicated that the majority of the schools, approximately 2,043, had not made AYP in reading.  This was the largest category in which we had not made AYP, although a substantial number of schools had not made AYP in math as well. 

We also asked about the number of schools not making AYP because of a single sub-group.  This has been an issue of debate over the last couple of years, and the answers varied by sub-group.  Approximately 70 schools did not make AYP solely because of their African-American students.  About 15 did not make it because of their Hispanic students; 32 did not make it solely because of their poor students, and 94 did not make it because of their limited English-proficient.  About 155 did not make it because of their disabled students and about 94 schools did not make it because of their 95 percent testing requirement.  This also means, conversely, that most of the schools that are in sanction in the cities are in sanction because of more than one sub-group at a time. 

Now, for the sanctions themselves, starting with choice.  The data indicate from our survey that the number of students transferring to a higher performing school under NCLB increased from 11,292 schools in these 36 cities in 2002-2003 to about 22,553 students in 2005-2006.  This translated into the huge leap of participation from one percent to 2 percent.  The percentages are still pretty low in terms of choice, but we did ask, in addition to NCLB choice participation, about the numbers of students transferring for non-NCLB reasons. 

Here, the numbers were much higher.  About 325,000 students, or about eight percent of the enrollment, actually transferred in these districts for reasons other than No Child Left Behind.  If one combines the two-percent No Child Left Behind plus the eight-percent non-No Child Left Behind to non-district charters and private schools then the portion of urban school students using some form of choice or another exceeds about 30 percent. 

We also have considered data on the length and numbers of windows during which parents have to choose the numbers and types of methods of communications with parents, the numbers of choices, school capacity and other information in this paper.  In general, the data shows some progress on each of these fronts since 2002-2003 - longer windows, more choices et cetera. 

Next, supplemental educational services.  Here, the data showed that the number of students participating in SES has increased substantially.  The numbers in the responding districts grew from approximately 110,000 students in 2003-2004, mainly the first year in which SES was really offered, to approximately 180,000 in 2005-2006.  About 16 percent of all eligible students are now served by SES. 

This next number is going to surprise, I suspect, everybody in the audience, but the data also indicates that approximately 95 percent of all students in these 36 cities now receive their Supplemental Education Services from a private provider, not from the district.  That is 95 percent.  Only six of these 36 districts are now able to provide their own services.  For the most part, urban school districts are out of the SES business.  Despite the apparent increase in the numbers of students receiving SES, however, it also appears that the rate of participation remains flat. 

The number of SES participants per school actually held steady at about 112 students between 2003-2004 and 2005-2006.  This suggests that the overall increase in numbers has been due largely to the higher numbers of schools that have to offer SES and not to the overall yield in the program itself.  We have not finished analyzing the data yet but we did look at the percentages of students served in the districts or in cities where the district could provide services and where they cannot; I suspect this will be of interest to the folks from the Department of Education. 

As I said, the average was about 16 percent; but in cities where the district was able to provide their own SES services, approximately 29 percent of all eligible students actually received services.  In cities where the district was not able to provide services, only about 11 percent of the eligible students were being served.  If all of the cities were able to provide SES at the same higher rate of 29 percent, then participation numbers would almost double by about 150,000 kids. 

Third, we asked about the strategies that districts were using with their schools in corrective action.  The most common strategies involved technical assistance, professional development, informing parents about the status of the schools.  The least common strategies involved charterizing, privatizing or giving the schools over to the individual states to run.  Finally, we looked at the strategies being used with schools in restructuring.  Here, the pattern of sanctions and the pattern of options is pretty much the same as we saw in corrective action; same kinds of things being used most often, the same kinds of things being used less often. 

There are lots of ways, of course, to slice all of these data but in some ways we are really getting started on our analysis.  We have considerably more data than we are presenting here or even in the paper.  I would like to wrap up here by making three quick points.  One, we are starting to review various local evaluations of SES and finding very mixed results; modest at best on average and harmful in some cases.  The effects of SES on state test scores really depends on the provider you are looking at. 

Two, we are getting better as cities at implementing the law but implementation, I have to say, is quickly becoming an exercise in compliance with rules and sanctions that are poorly designed to raise student achievement. 

And finally, let me just make one quick observation about the cascading nature of the sanctions themselves.  This cascading process does not seem to be very effective.  It has school districts chasing an annually changing set of strategies without enough time to make any of them work.  We are making, we think, very good faith efforts to make this law work and we continue to support it.  But, in the end, Congress is going to have to recalibrate it in order to produce the results that they want.  Thank you.

Frederick M. Hess:  Jeff?

Jeffrey Henig:  Rick and Checker pushed us very had to do PowerPoint; it is not my métier, so we will see how this works out.  It is already showing.  So I’m going to talk today about supplemental education services, and Mike already provided more data than I found looking.  This is a research-thin area, as he said, but nonetheless all of us have some memory of being in school and there was a kid there who was on the margins, always there; people vaguely knew who he was but no one remembered his name and I want to suggest in some ways SES is like that.  In the run-up to NCLB and, subsequently, the popular kids are disaggregated test scores, adequate yearly progress, highly qualified teachers.  People have not talked much about SES - I’ll show you some data on that - and really did not think very much about SES leading up to the legislation [indiscernible] that much subsequently. 

Very quickly, the basics on the program are familiar.  SES provides out-of-school tutoring, meaning tutoring in the afternoons after school, on weekends, on the summer.  It is tutoring for low-income students in failing schools, Title I schools that have failed to make adequate yearly progress.  Providers can be the districts but the law stipulates explicitly that the providers can also be both for profit and nonprofit providers.  The states are responsible, according to law, for approving providers and for monitoring their performance.  The funding for the program comes from the district’s Title I allocation, and districts are required under the law to set aside at least 20 percent of their Title I allocation to cover the cost of combined supplemental education services and any transportation costs associated with the choice provisions of the law. 

I think it is fair to say that at the time this was being talked about, again, the choice provision got way more attention.  I think most people involved expected that the choice part of the program was going to be a bigger chunk than it has turned out to be compared to SES.  By any measure, SES is a major policy initiative.  Compared to choice, it involves more children; in 2004-2005 nationally, over 430,000 kids.  It is a big-bucks industry, potentially; up to $2-2.5 billion a year, conceivably.  This is not what is actually being spent now, but what would be the case if you sort of maxed out that 20 percent on Title I.  And by the way, the 20 percent is not a ceiling.  It is a ceiling in terms of what the districts are required to provide but it can be more and the program is growing rapidly. 

The program is growing rapidly, over 210 percent between 2002 and 2003 and 175 percent between 2003 and 2004.  It is a major initiative also because it represents a new kind of policy.  In some ways, this is at least interpreted by some people as the government endorsing the idea in an era in which privatization of social services is the scrimmage line for partisan battles between Republicans and Democrats, between the Right and the Left.  SES is an endorsement, at least in principle, of the notion that for-profit providers may be able to do a better job at providing basic services than government employees. 

My paper suggests that this is a program that is better understood in many ways as the outgrowth of inter- and intra-party maneuvering rather than the culmination of a deep and sustained thinking about how kids learn, how families and schools interact in communities and how reform tools take root and succeed.  In served in an intra-party sense different functions for Republicans and for Democrats. 

For Republicans, what was appealing about SES was that it was a way to assuage conservative Republicans who were distressed over the Bush administration’s backing down on insisting on a voucher component in the law and SES as stepping in as kind of the replacement for choice had three benefits.  It looks like vouchers in some ways.  It is portable aid that families can carry with them from provider to provider in principle.  It is given to individuals and not to schools and it involves the private sector. 

It is also a program that provided a nose under the tent, if you will, for profit providers, a growing industry and one that many conservatives wanted to encourage.  And it gave in just the bargaining and political landscape around the law; it gave conservatives on the further Right end of the party, a chance to claim that they have had an impact after they, in fact, felt somewhat marginalized by the elimination of vouchers. 

The program served a different function for Democrats.  For the new Democrats, if you will, Lieberman and others, it was a chance to demonstrate that Democrats are not anti-market, that Democrats can and will pragmatically employ market solutions when they make government work better.  For the old Democrats, if you will, the traditional mainline Democrats – the Kennedys, Miller and others, the program had three virtues. 

First, and most importantly, it was not vouchers; secondly, SES because it is delivered out of school, it did not represent a direct confrontation with the teacher’s union encroachment on their turf; and third, SES was in many ways a substantial expansion of public responsibility rather than a contraction or shrinking of public responsibility and a new way to get aid to Democratic constituencies’ low-income kids, minorities, central cities. 

I mentioned before little attention.  I analyzed the coverage in major newspapers, both of NCLB, generally, and SES in particular over time.  This just maps out whether… the red on the top is whether NCLB was mentioned anywhere in the text of the articles.  The blue is whether it was mentioned in the abstract, an indication of a more substantial focus, and the two lines on the bottom are for SES.  And you can see, if you take being mentioned in the abstract of the article as being an indication of serious interest, there is a little blip of interest around 2001, 15 articles, really minor and fell off rapidly. 

Jeffrey Pressman and Aaron Wildavsky, political scientists, have made the argument that policies imply theories, by which they mean that every policy, at least, implicitly, has a network of assumptions about causality, that if you press here, the following things will happen; this will lead to that this will lead to that, and so on.  For SES, you can look for the theory but there is not really one.  And instead, I suggest in this paper there are two simplifying images that in some ways animated the actors, and in the case of obsolescent localism worked similarly for both Republicans and Democrats as an analysis of the problem that needed to be addressed. 

And the market metaphor - I will say more about this and second - was a clearer notion of how SES would fit into the solution in a way that was intellectually consistent with the Republican agenda, generally, and with the thinking behind the law.  So obsolescent localism, this is not a well-honed thesis but a collection of impressions based on demographics, socio-political trends that some people have come to believe are associated with higher mobility in a more contemporary American society that is weakening people’s attachment to localities, weakening people’s attachment to place and making localities in some senses structurally weaker.  The image of obsolescent localism suggests that policies at the local level tend to be driven by parochial values, interest group politics, over-bureaucratization and reflexive loyalty, the status quo. 

Now there are two different kind of images that Democrats and Republicans draw on in thinking about obsolescent localism, I suggest.  For Democrats, there is a history of thinking of localism in part in terms of Southern resistance to integration, to Brown versus Board of Education, and in a more contemporary vein, to suburban exclusionism, suburban tendencies to retreat into their localities, put up high barriers for entry, to take care of their kids and their kids’ schools but make it tougher on inner cities to deal with the folks who are left. 

For Republicans, the image is more one of Union-dominated bureaucracies in large urban central cities.  In both cases, the predictions, if you will, that seem to emerge out of this image of obsolescent localism is that we should expect the locals to resist SES, to resist encroachment on their control of their Title I monies, that there will be a need for strong federal and state role to enforce this to make it happen and then over time the locals will be gradually displaced at the local districts as a major actor.  The market metaphor very quickly assumes that what this program will do along with choice provisions is turn families into consumers who are free to shop around for providers who will best serve their kids.  This will create a competitive environment in which various providers will up the ante in whether to gain greater market share. 

For some supporters of this notion, that includes the notion that the district will up the ante in terms of its performance as well.  The districts will just be one under this model among a diverse array of tutoring providers; and there are predictions that evolve out of the market metaphor.  One is that there will be an active demand, that there is a latent demand for tutoring services that will be activated by the addition of money vouchers that… essentially that families will be able to use, that there will be a vital supply-side response, that new providers will come out to engage in this market and that there is an edge for profit providers. 

Very quickly, since I’m out of time, what do we find?  I want to argue that what we find is something quite different from this.  The local districts are still the fulcrum of activity.  Rather than resisting outright, some districts have embraced SES, that there is a slowly growing demand side but it is a tough sell.  Getting parents to sign up and getting kids to go is much more difficult than many anticipated.  The paper includes an analysis of who the providers are.  Most of them - there are some 17,000 - are local, parochial providers if you will, Mom ‘n Pop social service agencies and the like who will operate in only one district or certainly in only one state; 93% by my count operate in one state only. 

I used the language “dual market” of these local Mom ‘n Pop providers versus a multi-state corporate provider.  Corporations make up only about 2.5 percent of all the approved providers but of the corporations, over 70 percent are operating in multiple states whereas in contrast, social service organizations make up about 13 percent of all providers, but only less than two percent of them are in multiple states. 

Finally, I argue the best way to understand this is of SES as a local contracting regime, like local governments contract for other services.  The locals are central; they are central not because they are central in the law but they are central because states and federal government lack the capacity to really get involved and monitor what is going on.  They are central also because locals control access to schools and teachers, and at least for the for-profit providers, it is proving very critical for them to get onsite provision or to get recommendations from teachers and to use teachers as providers for reasons I’ll talk about in the question-and-answer period because I’m already in deep trouble.  But I want to suggest in this contracting regime, access depends on not just demonstrated performance but traditional things that count in a lot of local government, which is political ties, support from active community organizations and the like. 

I conclude with some implications for policy and you can read this in the policy.  I’ll just summarize it in two sentences and that is the response can either be make the program fit the original image, get tougher on the locals, beef up the market sector, or work with the locals to improve their capacity to do this well and do this right; I side for the latter.

Frederick M. Hess:  Thanks Jeff. Paul.

Paul Manna:  Hey, good morning everybody.  My paper focuses on the importance of the state’s role and state implementation of the No Child Left Behind remedies, and I think this is an important focus for this conference for this project.  A lot of people correctly state that NCLB is a major law.  It has expanded Washington’s reach across the country into the nation’s schools, but I think in saying that it is important not to overstate the federal role. 

State policy in education remains absolutely crucial, and in the paper I used the phrase that often I think of state policy as the fuel that powers the NCLB engine, that if it were not for the states taking action, making policy decisions, et cetera, NCLB would basically go nowhere and the law really relies a lot on what is happening at the state level.  Further, if you look across the states, states are diverse places in terms of their educational governance structures, in terms of their political climates, and also in terms of the bureaucratic capacities that they have in order to make laws work. 

This is true in education; it is true in all kinds of other policy areas.  And so as you look across the country at how NCLB is being implemented in the states, it is not surprising that you see a lot of variation when you examine the ways that states have tried to help and do their part in implementing the NCLB remedies.  So in my remarks to day, what I want to do is just briefly describe some of that variability that I have uncovered at the state level and then also discuss the impact of that variability. 

What impact does it have on how the law is unfolding?  And the first issue I want to start with is timing.  As you know, state testing systems and accountability systems are the major instruments that are used to determine whether school districts and individuals schools have made adequate yearly progress.  In most places, students take their tests in the spring, maybe March, April or May, typically; and so it is crucial for NCLB and its remedies to work.  It is crucial for the states to swiftly and accurately process all those results so that people at the local level know which remedies are supposed to be implemented and so that those people have enough time to implement them. 

Today, as this first slide shows here, actually, though, nearly five years since NCLBs passage, the states still struggle to determine and report out final AYP results on time.  What you have in this figure which appears in the paper, on the horizontal axis, on the X-axis, you simply have the rank-order of the states when they release their final AYP results.  And so the first state is the dot farthest to the left and then as you work your way across, the last state is the farthest to the right; and then on the vertical axis there you have the day at which the final AYP scores were released.  If you look at that, you see that most states are releasing these final numbers sometime during the month of August, this window, almost right as the school year is beginning. 

In several states you can see it is bumping right up against late August, early September; some states, even after the school year has begun.  I think if you count the dots there it is about seven or eight are getting these results out the door.  If you are a quick study, if you maybe count cards in poker or something, you will notice that there are only 43 dots on the figure, and that was because as of late October 2006 when I gathered these data, there were still seven states that had not completed the final assessments of AYP that would affect the current school year.  I have to go back and see if they are done since then but when this was put together less than a month ago that was the status. 

Clearly, there are big implications for implementing the law’s remedies.  If you are a parent and you want to opt for school choice or you want to try to get your child enrolled in supplemental services, it would be great for you to know that you have those options in advance of the school year.  I mean, it is hard to fault a parent who would not want to uproot a child’s routine, et cetera, early in the school year if the parent is only learning this information right on the eve of the beginning of school. 

Further, think of the role that local districts play in implementing the remedies and especially some of the large urban districts that Mike talked about.  If there are remedies that need to be put into place that involve more dramatic action, the corrective action or structuring kinds of things, imagine having all that on your plate or learning about that at the same time that you are simply just trying to get school started.  If anyone in this room has been a teacher (I was a teacher before entering this life), you know the start of school is a hectic time and those last two weeks of August go like that.  So to have this additional knowledge coming in as you are trying to get the regular year started can be quite challenging. 

Let me talk next about supplemental services.  I’ll talk about what Jeff called “the unpopular kid.”  I’ll give the kid some stage time here.  This is one other area that I examined in the paper, and as you know and as Jeff briefly summarized there, the main state responsibilities for supplemental services, there are really three things involved, right?  States are supposed to develop a list of supplemental services providers, the approved providers.  They are supposed to disseminate information about those providers so that local school districts know who they are and so, importantly, parents know who they are.  And also the states are supposed to hold providers accountable for results. 

If you look out there at the provider list the states have put together -and this is one thing I analyzed systematically for the paper - I got a hold of all of the supplemental services provider lists that the states have put together.  So the document that a local school district or parent would see when it was trying to decide which provider to choose, I systematically analyzed those lists, looking for 21 specific provider characteristics; and when you look in the paper there is a logic to the 21 characteristics that I looked at. 

I’m just showing you a handful of them here just to give you a flavor of what is in the paper.  Here is just a quick summary of a couple of main points that come out and these particular elements illustrate the point.  First of all, most of the state provider lists do a pretty good job of giving sort of basic information about the provider - a phone number to call; obviously, you know the name of the provider; as you can see up here, the geographic area that the provider serves, right?  Some providers claim that they will give services anywhere in the state; others say “We are only going to serve this particular school district or community,” and the vast majority of the provider lists actually do contain that information. 

I was surprised though that not every state SES provider list has a phone number for the provider.  Not all of them do that; it is over 80 percent.  About 85 percent have the phone number but not all of them.  I mean, that seems like a very basic piece of information that you would want to have in there if a parent is going to take advantage of this kind of remedy. 

One area where it is clear that the states struggle is on the issue of holding supplemental providers accountable for the results, and if you see the sort of three lines in the middle there on the overhead, it suggests that in these provider lists, parents get very little information about how their provider is going to be reporting results back to the parent or the school, how often that will happen, and if there is any evidence of the provider’s tutoring services having been effective. 

What kind of data do they have to show that their work has been effective?  Now, that part is particularly troublesome if you think that one of the main things you would want a parent to look at when choosing a provider is whether the provider helps the child to learn material, helps the child to catch up.  What research base is there for that and how is the provider held accountable?  There really is not much there in explicit terms.  What most states -- or I guess not most; actually about half of the states do provide however - and that is the last line up there - they provide an opportunity for the provider to insert sort of a narrative description of the program. 

And so you will often see sort of some nuts-and-bolts info and then sort of a paragraph that the provider describes itself.  Oftentimes those things do mention some of these issues about accountability and evidence of success but it is not presented systematically.  So imagine if you were a parent and you are trying to pick a provider, do you actually want to wade through maybe 30 or 40 paragraphs to try to spy on the ones that actually mention the accountability issue?  That might be a very difficult task, and like I said, most states do not actually single that information out and present it. 

Okay, the last thing I want to talk about is corrective action and restructuring those remedies.  Really, one of the things that I found, and that I think others in the panel here and other panels have found, is that gathering systematic state-level information on these two remedies in particular is very difficult.  To know exactly what is happening across the states is hard to know. 

Here are just some examples of things that in my own work it has been difficult for me to discern.  You would think that at the touch of a button, you would hope that at a touch of a button, five years into NCLB, we would know this information, but we actually do not.  This is really crucial, especially if you think about the options that are embedded within the corrective action and the restructuring remedies.  It is bad enough that we do not really know how many schools in each state are at each level of improvement or at these levels of improvement in particular.  If we want to know how these remedies are working, what we would really want to have is information on which particular flavor of restructuring has been implemented sort of most consistently in this state compared to this state. 

What are the different remedies?  Because the remedies, like Jeff said, they are the levers that policy-makers think we can pull.  If we pull this restructuring lever in this way, we will hopefully get an outcome that we like.  But right now we do not even know systematically across the states which levers are being pulled in which areas. 

So just let me wrap up with a few brief points.  The first one is, again, I think it is important for us not to over-state the federal role in the nation’s schools; even though NCLB is a major law, a lot of what happens with this law depends on state choices and, as Mike was saying earlier, local choices.  So the law is still unfolding and being implemented in a fragmented system of governance that we have for American schools across the country. 

And so given again these internal political climates in the state policy choices, et cetera, those kinds of things will continue to have a tremendous impact on how the law’s remedies are unfolding.  So thank you very much.

Frederick M. Hess:  Thanks Paul.  Mike?

Michael Petrilli:  Well, I was going to start by saying that I was the kid in Jeff’s class who was going to get in trouble for not doing what he was told because I do not have a PowerPoint presentation.  But at least when I get put in time out I’ll have some company - Jeff would be joining me and that makes me feel better. 

As Rick said, that is popular in Washington today, to declare support for a policy but complain about its implementation.  We certainly see this in the Iraq debate where even the neo-cons in this very building who advocated for the war and for President Bush’s vision of establishing a beachhead for democracy in the center of the Middle East are now complaining that the mission has been botched.  And, of course, we also see it in the debate over President Bush’s most ambitious domestic policy, the No Child Left Behind Act, which tries to establish a beachhead in the battle to close the achievement gap. 

Now, members of Congress and advocacy groups and think-tanks who support No Child Left Behind have been increasingly vocal about its implementation.  For example, recently, Senator Ted Kennedy, soon to be chairman, recently said of No Child Left Behind:  “The administration’s implementation of the reforms has been inadequate and ideological.  Its ineffective implementation has undermined the reforms it said were so important.” 

Now, what all of these people seem to be saying and believing is that with the right people calling the shots in the US Department of Education, making good decisions and acting wisely, the law could work as intended.  So in other words, when President Clinton comes in to office in 2009, and she appoints Secretary Broad or Secretary Bersin or Secretary Haycock - I could keep going but I’ll take charge of implementing the law, all will be well. 

Now, this argument, of course, has huge repercussions; if it is true that all that is wrong is implementation and, especially, federal implementation, then the law itself does not need many changes.  So exploring this contention was the purpose of my paper, at least for some of the law’s remedies. 

Now, I should admit upfront that I am not exactly an unbiased outsider.  I served for four years under Rod Paige and helped to implement some of these policies; and since leaving the administration I guess you could say I have been a somewhat vocal critic of some Secretary Spellings’ decisions.  So keep all of that in mind when you consider my basic conclusion which is this:  At least when it comes to public school choice and restructuring, I have come to believe that this law is basically unimplementable.  So let us get to that.  So in the paper I looked at three policy paradoxes, the dilemmas that the Department of Education has tried to unravel, has had to try to unravel. 

Number one, in order for the public school choice and free tutoring provisions to work, local school districts must take aggressive actions to inform parents of their options, yet districts have little incentive to do so; number two, students and schools in need of improvement are to be provided with options within the same school system, but in many big city districts there are not enough good schools to go around.  And paradox number three, school districts are supposed to restructure persistently failing schools, yet these districts rarely have the inclination or the political will to do so, and some loopholes in the law make this bold action even less likely. 

I do not have time to discuss all three so I’ll focus on the first and consider what the department tried to do to address it and why it has failed thus far, and this is informing parents of their options.  Now, it almost goes without saying, in order for parents to take advantage of school choice programs, they must know that they exist, and this is a huge challenge.  After all, we know that corporations spend billions of dollars trying to get us to buy soap or cars; cutting through the fog of information, especially on a shoestring budget, is never easy. 

This is even more the case when the target audience is poor and overworked and, perhaps, even overwhelmed by life.  Doing it right means going into the community, knocking on doors, making announcements at churches, posting flyers everywhere, showing up at festivals, blanketing the airwaves, and on and on and on.  In other words, it is a whole lot of work to do this right.  So under even the best conditions, informing parents of their options under No Child Left Behind would be difficult.  But of course the construction of the law creates a unique problem; it requires the districts to inform parents of this option, yet doing so is at odds with their own interests. 

Again, many big cities do not have any school choice options to offer to parents and when it comes to supplemental services, few are all that interested in having parents go to private providers or at least because the law is not “use it or lose it” in terms of the money for choice and SES, but use it, or if you do not you can use it for whatever else you want to use it on.  There is not a lot of incentive for districts to make sure parents are using these services. 

Now, these problems were easy to predict as soon as the law’s ink was dry.  In fact, at the last conference Rick and Checker held, these were all raised.  So what did we do to try to address them?  The first thing we did was to try to appeal to district’s better angels and to encourage them to do the right thing.  So for example, we published these very colorful innovations and education guides, 50,000 each sent in to all the school districts in the country, held conferences, talked about concrete things districts can do to inform parents. 

What did we learn from other school choice programs?  What have other districts learned from their own school choice programs?  But let us be honest; it was not really for lack of know-how that most of these districts failed to inform parents effectively.  Again, those pesky perverse incentives and there own struggles bureaucratically to inform parents really about anything had not gone away.  So we adopted a second strategy - if you cannot work through the districts, work around them; in other words, empower the outsiders. 

Through the Secretary’s discretionary fund, we made some grants to several advocacy organizations dedicated to informing parents of their options such as BAEO and CREO.  So BAEO, for example, launched an aggressive outreach campaign in cities like Detroit, Atlanta, Philadelphia.  They used a mix of radio ads, grassroots communication and media relations. 

The project had some success.  Knowledge of the NCLB options increased in their target cities from 37 percent to 72 percent over three years.  But these funds for these kinds of projects were quite limited, could not have much of an impact.  That is when we discovered the parent information and resources centers, parent information and resource centers program or PIRCs.  This certainly sounded like a program that was about informing parents of their options.  It actually was not, but that was okay; we kind of used some application language to get these PIRCs to do the work of informing parents of their options and soon about 70 of these organizations around the country were doing so.

 Now it is hard to know whether any of these non-traditional methods going around the districts worked and even again in such a big country with so many children eligible, even these activities have to be seen as mostly symbolic.  Now it soon became clear to us that what mattered most was whether districts bought in to the choice and tutoring provisions and decided themselves to launch aggressive outreach campaigns; after all, they have particular advantages. 

They know who exactly is eligible for choice and tutoring and nobody else has that information.  They have the power to send information home in students’ backpacks or, even more important, to instruct principals and counselors and teachers to tell parents about these options at back-to-school night or at report card night. 

And as our innovations guides found out when we talked to districts, it became very clear that parents mostly trusted information that was going to come from their teacher or their principal.  So with the arrival of Secretary Spellings, the department tried a new tack.  It tried to replace some of the law’s perverse incentives with different incentives, those that would encourage districts to play ball.  In other words, it said let us make a deal. 

It launched two different pilots; first in August 2005, it allowed four districts in Virginia and later in some other states to flip-flop the order of public school choice and supplemental services.  And then, second, four urban districts that were in need of improvement were allowed to provide the tutoring services directly.  In both of those cases, the deal was you have this new flexibility; in return you need to do much more aggressive parental outreach and show that you are getting better participation rates. 

Evaluations on those programs are not due until February at the earliest, so we do not know yet if they are working.  And I think as some others will argue, allowing these districts to serve as tutoring providers could have some other deleterious effects, though I know Mike would disagree.  But no matter what the impact in this handful of cities, still the small pilots are not going to have much of an impact on the larger national picture. 

Now, one strategy, of course, the department has not adopted is getting tough with wayward states and districts.  Now, it is not necessarily true for the law as a whole.  There have been instances when the administration has taken away money for states or for districts for not implementing parts of the law.  So for example, states that are not testing new teachers as is required under highly qualified teachers’ provision or there is not testing LAP students as required.  In some cases, the department has withheld administrative funds. 

So why did we not take the same action when it came to choice and SES?  The answer is pretty simple; the examples above are pretty much black-and-white.  This is a matter of gray.  In almost all cases, districts were following the letter of the law.  They were sending letters home to parents.  Those letters might have been full of jargon and written in eight-point type and really done everything they could to discourage parents from taking advantage of the choices, but they existed.  And at the end of the day, going through the motions is not illegal.  In other words, the department did not have legal grounds to take action. 

All right, so let us recap very quickly.  None of the department strategies appear to have made much of a dent, though I would argue we tried to be relatively proactive about trying to make this work.  So let us return to the original question posed by the paper:  Was implementation, especially federal implementation, the problem? 

Now, to be sure there have been mistakes.  We could have moved faster to model the kind of parent information and outreach expected from districts or worked harder to do something about getting states to release the information sooner.  We could have issued a regulation that would have made it harder for districts to roll left-over money intended for choice and tutoring into other programs, creating stronger incentives.  And we could have made an example out of some districts, if we could find them, that were clearly not following the letter of the law. 

Now, these actions might have helped at the margin; I do not think they would have changed the basic story line though because as far as I know, nobody inside or outside the administration has figured out ways to fundamentally solve these problems.  So I think it is hard to argue that implementation, or better implementation, is going to really change the story.  And it does not, though, mean that changing the law is an answer either, though I’m happy to offer a few suggestions when we get to Q&A. 

The hard truth is that there may be no solutions to these problems because they are inherent in our federal system.  If there is one lesson policy makers and all of us should take away it should be this:   While it is hard to get recalcitrant states and districts to do things they do not want to do, it is impossible to get them to do those things well.  So in other words, we can coerce districts and states to follow the letter of the law but we cannot coerce them to follow the spirit of the law.  And when it comes to these things like informing parents, creating new schools of choice and overhauling failing schools, going through the motions is not good enough.  Thank you.

Frederick M. Hess:  Thank you Mike.  Checker?

Chester E. Finn:  Well, in order to say something, let me thank Rick and his terrific team for pulling this conference together as well as they have done and also our very fine authors for yeoman work on these papers.  As I re-read the papers for this panel, and most of the papers for the other panels, I began to think that instead of handing out coffee and sweet rolls over there, we should be handing out mood-altering pharmaceuticals, the kinds that deal with depression, because, fundamentally, this collection of papers tells a fairly depressing story five years out. 

It is fair to ask whether five years out is the right timeframe in which to begin to pass judgments, and it is five years.  If you remember, it was those crazy months after 9/11 that Congress finally returned to and buckled down to and finished work on the No Child Left Behind Act, which was in late 2001 and it was then signed into law in early 2002.  So it is five years out.  We know that any large, complex, ambitious federal law requires fine-tuning and revision and is not going to work perfectly out of the gate. 

That I think goes without saying but I think the question as we move into reauthorization season for No Child Left Behind, and we have no idea how long that season is going to last, but the question as we move into reauthorization season…I do not have any answer to this one today but the question is can the problems be fixed through amendment?  Or are we, as Mike Petrilli just suggested, dealing with something much more fundamental that goes to the very nature of American educational federalism and a kind of large over-reach by the federal government aspiring to accomplish something which the federal government is actually incapable of making happen no matter how it writes the statute. 

I think that is the question and if you go back to what it was we were trying to accomplish here, what is the country trying to accomplish, let us do keep in mind this was because kids were not learning enough, are not learning enough; poor kids especially are not learning enough.  Achievement gaps are way too wide and a lot of schools are not every effective in solving those problems is what this was all about.  So the appropriate criterion against which to judge all this is whether those problems are getting solved, not just how our various sort of programs working in a kind of bureaucratic or implementational sense. 

There is a little bit of good news in terms of student achievement among poor kids in the country over the last five years but it is really unclear whether it is attributable directly to No Child Left Behind or perhaps to the spirit of standards-based lifting all boats that No Child Left Behind is part of across the country and that actually pre-dates and may survive No Child Left Behind.  There is not a lot of progress, however, in getting schools and districts in need of improvement off those lists.  In fact, Mike Casserly’s paper makes pretty clear that the number of schools and districts on those lists of needing improvement keeps growing, not shrinking. 

There is incredible variability by state, a point made not just by Mike Casserly but also by Paul and Jeff.  America’s basic public education governance structure explains why that is the case but in the context of a national law, it actually seems sort of insane that there is this much variability across the country by state.  There has been very little use of the public choice program for a host of reasons though, interestingly, there is considerable use of other kinds of school choice in American education today. 

There are a variety of reasons; one that pervades almost all of these is how late in the year the NCLB results come back.  That goes to a different problem which I do not believe any of our papers even gets to, which is the complete inadequacy of the American testing industry and testing enterprise to anywhere near live up to the challenges of speedy, accurate reporting that this law presumes will happen, and that has to happen in order for this law to work as intended.  The SES utilization is greater than public school choice but it, too, is plagued by late notification as well as by the kind of underlying confusions that I think Jeff’s paper unpacks as to where did this come from and what is it supposed to do. 

The corrective actions and restructurings, I think it is clear from everybody’s papers, have been relatively mild.  Mike Casserly uses the word “softer,” kind of friendly interventions at both the school and district level, not drastic, dire, disruptive interventions.  Why is this working so unevenly, clumsily and for the most part ineffectually?  Well, part of it is, no doubt, ambiguous and loophole-riddled statutory language; that, at least in principle, could be corrected through amendment. 

Part of it certainly is implementation at the federal level and everywhere else.  But I really think the overriding problem, and Mike Petrilli was there at the end of his comments, is the federal government actually lacks the leverage to effect the changes contemplated by NCLB except in situations where states and districts want to make those changes, actually want to make those changes.  In other words, the law itself is a huge over-reach, given the limits of the federal government’s ability to alter the behavior of states and districts in America, leading to Mike Petrilli’s word, “unimplementable.” 

If we were living in France or Singapore or a variety of other places we would be having a completely different kind of conversation; then we really would be able to focus mostly on whether kind of the central administration of the Ministry of Education was doing its job right.  But when you are living, when we are living in as loosely coupled a federalist structure as this one, with as little leverage as there is coming from Washington and as many other moving parts at the state level, the district level, the school level and then all those other things, like for example the testing industry, it will all have to kind of work really well and work in harmony and work in sync with each other in order to change.  And this is an enterprise famously full of institutions and places that are kind of set in their ways and are not really keen to change - you can begin to understand why this is not working. 

Let me quote from Paul’s paper because I think he says this really quite crisply in his paper:  “Absent a major revolution in educational governance in this country, both federal and local officials will continue to rely upon state policy and leadership to improve the educational fortunes of thousands of American children who attend struggling public schools.  The continuing persistence of state and local control of education means that NCLB’s ambitions and federal officials’ promises to strictly enforce the law continue to collide head-on with the primary institutions that control American schools.” 

Jeff’s very insightful case study of the political economy of the SES program helps explain sort of how it came to be that Washington lawmakers created a program, and that is one program within the law, of course, that probably cannot work as intended.  He writes of the SES program’s “thin and tangled intellectual roots,” his phrase.  His paper, in fact, reminded me a little bit of Pat Moynihan’s famous book about LBJ’s community action program titled “Maximum Feasible Misunderstanding.” 

So many disparate notions and expectations and theories came together in one program, sort of co-mingled there by an over-ambitious or accommodating or compromising or naïve Congress.  So many disparate notions came together; there was actually no possible way that the resulting program could in reality be implemented to the satisfaction of any one of those theories or any one of those interests.  In NCLB as in community action, I think it is fair to say successful implementation would require the active and sophisticated coordination and subordination of many, many different adult interests and power centers and roles. 

And yet it is not really in their collective interest… it may be in their collective interest, but it is not in their individual interests to make this work.  The collective benefit would only be that of children, and they, of course, do not have power centers.  They do not have lobbyists.  They do not have bureaucracies.  There is no children’s bureaucracy in America.  There are only adult bureaucracies and it is those that would have to change in order for the kids to benefit.

I do not think anybody in this whole equation has the leverage within our system to actually make change, force to change, cause to change if they do not want to change.  If they do want to change, this can go great, and there are places around the country where it is going pretty well.  But there are places that for their own reasons want to do it right, want to change, want to do it differently. 

So let me just finish where I started.  I think the big question, and I hope we can return to this at the end of the day in the closing panel where we have Kati and Mike, Diane and others up here, I think the fundamental question is whether the problems being surfaced today can be solved through reauthorization, or whether they go to something far more fundamental than anything Congress can or is likely to do as it amends this law over the next few years?  Thank you very much. 

Frederick M. Hess:  Jeff, let me go to you to kind of start the conversation.  In some sense, your paper ends in a more optimistic note than Mike’s, particularly when you talk about capacity building or investing in local ability to make SES work.  You actually seem to suggest there is more room for refining this thing than Mike or, say, Paul’s paper seem to suggest.  You want to speak about that?

Jeffrey R. Henig:  Yes.  I do not want to be a sort of an apologist for all elements of the legislation but I do think it is important to think clearly, as Checker said, about the difference between evaluating an immature and a mature policy regime.  I think there are many instances in which federal government or State governments have overreached and, by overreaching, have kicked into action responses that ultimately play out to the benefit.  And I think there is potential … just speaking in terms of supplemental education services. 

So part of the problem is lack of infrastructure and data management.  Part of it is lack of capacity in terms of understanding how to evaluate provider performance, things like that.  I mean, there is a whole history - and Paul has written about it well in his book - about how federal governments have borrowed capacity from the lower levels of government; and so the fact that the Feds cannot do it themselves I do not think is definitive. 

There is just one other point which is one thing that I think the law could do better, but there is an element of it as well, is thinking about how do you actually improve the political environment at the local level in ways that lead to greater willingness to try to take advantage of the positive aspects of the law?  I think the Federal Government from Washington can do that sometimes.  There were elements of the community action program and things like that in terms of public meetings and public information requirements that empowered groups that were somewhat disempowered. 

I think the disaggregated data aspect and the public test board aspect of the law has that capacity and probably already has that effect on the ground; that it is a new weapon for local groups to use to bring pressure on the local districts also.

Michael Casserly:  If I could pick up on one thing that Checker mentioned, I think he is absolutely right about this kind of issue about the said cohesion of the law.  I mean, if you think back to when this thing was first authorized, in many ways what really happened here… and anybody who has ever worked at Capitol Hill before kind of understand this dynamic. 

In an attempt to get a pretty bipartisan bill, both sides, Republican and Democrat, had a number of different options on the table that they wanted to put into the legislation.  But rather than thinking through how all of these various pieces would fit together, everybody simply said, “Yes, let us take all of them.”  And consequently, nothing locked together in any coherent way and it, I think, ultimately became the seed of a very unimplementable program. 

Second thing I would mention though that we have not mentioned yet here is the law really in many ways, both from its beginning other than its grand intent about raising achievement and closing achievement gaps, never had any theory of action behind it around how to actually improve student achievement.  And there really was very little capacity in the U.S. Department of Education about how the goals of NCLB might be met.  And there is certainly very little capacity in the individual states about how to raise academic performance. 

So in some ways, I think what ended up happening was that much of the action at the federal level and at the state level and sometimes at the local level, too, really became about compliance with the nuts and bolts provisions of the law and not about what it would take to actually raise student achievement, an issue that we thought the legislation was supposed to be about but is hard to find at any place in its implementation.

Frederick M. Hess:  There is an obvious irony there, which was the entire logic behind 1994 reauthorization—people were concerned that ESEA had become a compliant framework, which was getting in the way of actually making these dollars help.

Michael Casserly:  And the response was let us do some more of it.

Frederick M. Hess:  Mike?

Michael Petrilli:  Well, I think what is important to understand about, especially the choice pieces of this, is that in fact, the theory on one hand was that it would be another hammer, another way to [indiscernible] certain behavior to break through some political barriers to making important change but the only people that could actually implement it and enforce it were those same people whose behavior we are trying to change.  And I want to be clear; when I say that these parts of the law are unimplementable, these specific parts, I did not say that about supplemental services. 

I am a little more optimistic there.  I certainly think that the testing and accountability framework is implementable.  Every state breaks out data by race; that is a huge and important innovation and I think it has had some good benefits.  The question is, what can we do to provide political cover to the local superintendents and, to some degree, the State officials who want to, in my opinion, do the right thing, push for a change, push through the political barriers to change without creating this overreach, without over-promising?  So when it comes to choice, for example, I think we have to perhaps now admit that, as compelling as it is to say every child in a failing school will be given a choice, we cannot deliver on that promise. 

So maybe we have to say, okay, let us think about a grant program that provides funds to school districts that want to do public school choice and then can be held accountable for doing it; it is only 100 of those districts instead of 15,000, but if they are the right districts we might get more leverage.  It is not as compelling, exciting.  The rhetoric is not as powerful but I think it would be more doable and, again, might get the results that we are looking for.

Frederick M. Hess:  Mike?

Michael Casserly:  This issue has been brought up before in lots of different forms about kind of the conflict between the sanctions in the bill and who has to administer them and what that conflict is all about.  And I think that adds to why this thing is so hard to do, and why its implementation is so slow.  But there is no particular reason why you could not take the sanctions in their cascading fashion and redefine them around instructional interventions based on good research, maybe make this a multi-year school or district improvement process in which you have to use various instructional interventions based on good research.  And then follow it at some time certain by your sanctions, rather than having this cascading, ever changing, hard to implement, poorly-timed set of things that just [cross-talking].

Frederick M. Hess:  So what would that look like?  What do you have in mind?

Michael Casserly:  We have not thought of it all the way through yet but we did suggest to the NCLB Commission that maybe what they could do is, in terms of the reauthorization, was, like currently, have a year's kind of planning period and then if you didn't make your AYP targets go into, say, a three-year school improvement or corrective action phase whereby the school district or the schools had to implement various instructional interventions, more scientifically-based curriculum, more coaching, better professional development aligned with the curriculum and the standards and the like, and include maybe choice and sub-services from the get-go.  But define them around the kinds of strategies that you would need to raise achievement. 

Then if you could not get … if you could not make your targets by the end of that specified period, three years, four years whatever it is, then think about either closing the school or reconstituting or something like that.  But it would give people the time to actually implement something before you started to phase in all of the sanctions and made them space it out.  It would also help you solve this problem that Paul mentioned about the late submission of data that would occur, basically, at the beginning of that cycle.  But then you would have three years or four years to adjust and to design your program, and the like.

Frederick M. Hess:  Though it seems to me that there are two kind of operating theses embedded in what we are talking about.  One is the capacity thesis.  So when you are talking about the three-year phase, then the notion is that the technical assistance is not working, that people do not have a clear sense of what constitutes an effective scientifically-based curriculum and such, and that giving them the three-year window and the additional lag time, but then, I guess, the second is the Petrilli thesis is, well, that is nice, but it is not a problem of capacity so much as design and will. 

Michael Casserly:  Now, I really take issue with that.

Frederick M. Hess: Okay.

Michael Casserly:  Every piece of data that we gathered suggests that the school districts have been making better efforts at providing choices earlier in the school year.  Opening their windows for longer, having rolling windows and the like, providing the information in more languages does not mean that it is everything that the choice advocates want, to be sure; it is not.  But on the other hand, it does not suggest that the school districts all over the country are deliberately dragging their feet to make this happen because, as I have pointed out in our research, the number of kids who are actually availing themselves of one choice option or another, NCLB or not, is really quite high. 

And one of the main problems for the NCLB choice, as Checker pointed out, was that its timing is after all of the other choices have been made and the parents have kind of the worst options and the fewest numbers of schools to pick from.  One way to solve this would be to sync that up a little bit better, but it does not signal that somehow the low participation rates are the result solely of resistance; they are not.

Frederick M. Hess:  Paul, so at the state level, help us think through these kinds of competing tensions.  Take, for instance, the identification of school year.  How much is the capacity an issue?  And how much of it is actually an issue of will—that states and/or testing companies could get these schools identified much earlier if they actually wanted to push.

Paul Manna:  I guess one of the things to think about in terms of capacity (and someone made a comment a minute ago that we wrote this law that was supposed to not be based on just sort of compliance and filling out paperwork and it was supposed to actually have real results on the ground) well, if you think about the thousands of people who work in the education industry, public sector across the country, for years and even still with lots of Federal and other grant programs, those people were trained to use James Q. Wilson's term, “critical tasks.” 

The critical tasks for people in those agencies were sort of compliance-oriented kind of processing.  So the key task for a state education department for years was make sure the money is going out to the schools and keep track of where it goes and then make sure it is spent correctly.  Do not necessarily worry about the result at the end but just audit the books to make sure it was spent the way it should.  And that is the reason why you had the invention of these pull-out programs Title I because it is a great way for a local district to prove that it spent the money on the right kids. 

Whether it had anything to do with learning was anyone’s guess at the time.  I think we now think that it really did not help necessarily at all [indiscernible].  And so to ask compliant people who are trained and have for decades been working in organizations that are designed to work on compliance-oriented activities, it is not surprising that that is where everyone kind of migrates to; they kind of migrate to their comfort level. 

And so it is hard.  Some of the state officials that I have talked with for this sort of commented on that, that they make the comment that it is hard for them to work with local districts sometimes because principals, even, are sort of the managerial types on the ground.  They are not instructional leaders.  They are not people who wrap their head around the curriculum and what is going to make the students learn.  Some do, but by and large that has not been how we have trained them in this country, and so we need a real kind of shift in those capabilities.  When I think of capacity, I think that is one of the main issues.

Frederick M. Hess:  Okay, Mike.  Then we will open it up to questions.

Michael Casserly:  I think one of the great ironies here is that NCLB was really at the outset billed as kind of a follow-through to the standards movement, kind of the next steps in the standards movement.  But the great irony here in lots of ways, in the standards movement again, was in a lot of ways about defining our work, about the results that we got and the outputs, rather than the input.  Ironically, NCLB has unfortunately devolved to a place where it is now about the process and not about the results so much.  It has become very compliant-driven in ways that the standards movement was supposed to transcend and did not.

Frederick M. Hess:  All right, let us open it up to the audience.  You will find Juliet over here and Rosemary over here with microphones.  Again, please wait for the microphone; please identify yourself by name and affiliations, since many in the room may not know each other, and again please actually ask a question or else I will truncate your remarks. 

Jay Green:  It is a little bit of cheating for me to go since I am going to be speaking later.  I am Jay Green of the University of Arkansas, but I liked all the analyses I heard.  But I heard this incredibly gloomy conclusion about the impossibility of the Federal government to do certain things.  And it seems to me that is inconsistent with other evidence.  The evidence is that the Federal Government actually does lots of things quite successfully, including in education, where it imposes things upon schools that they do not want to do. 

One example I can think of is special education.  How was the Federal Government able to succeed in getting local districts to provide services to disabled students when previously they did not always want to?  How were they able to succeed at that but they cannot succeed here?  And that is my question.

Frederick M. Hess:  Okay, so Petrilli then Casserly.

Mike Petrilli:  So I guess I would ask Jay if he thinks that schools are providing services to kids with disabilities as well.  I mean, obviously that is an enormous compliance framework, and as Mike said this is now supposed to be about results.  But the trick is… I wrote a piece a while ago about how No Child Left Behind includes the what-works and the whatever-works theories of action. 

And the whatever-works people say, “Look, we are just going to focus on results and get however you want to achieve them,” and that implies that people could get better results but they do not want to.  And they do not want to because of their own self interest, political interest, the bureaucracy, the union, et cetera et cetera, but that we are going to change that dynamic by having some kind of hammer. 

I guess what I am arguing is that it is clear that the choice hammer is not working because the people that are supposed to pull the trigger on that are the same people we are trying to change the behavior of.  I am more optimistic that the accountability hammer can work if done right, but that is still an open question mark.  So if you want to go to the special ed. model, I think that is all about compliance.  I think what is tricky is how to make the results-based accountability work in this system and who is it that is going to pull the trigger.

Michael Casserly:  I will make my comment real short, Jay.  I actually end my paper optimistically, and I think NCLB can work, could work, if we do it right, and in some ways there was really something of a paradigm change.  A lot of people have talked about what NCLB was supposed to represent.  I guess I’m not terribly surprised in the grand scheme of things that we did not get it right the first time out, and that it is going to take a couple of more iterations before we really fine-tune this in a way that is effective.

Frederick M. Hess:  Paul?

Paul Manna:  Just one quick thing.  I think one thing that IDEA does is it empowers parents and their lawyers to get the things from school districts that, maybe, the districts might not be willing to apply and to provide.  And in the NCLB context, one thing that it seems like the Federal government could do a very good job of if they are really committed to it would be the information provision issue.  The fact that now we see breakouts of test scores by student subgroups is a powerful thing.  We see now that the Richmond public schools in Virginia are doing better with African-American kids than the Fairfax County Schools in Virginia.  That would have shocked people I would say 10 years ago before we saw the numbers. 

And so if the Federal government could take seriously the information gathering and dissemination function so we could know where these remedies are happening, that would be a really powerful thing that I think it could do which sort of teeters toward what I think Rick has called “suggestive accountability” in another context.  It provides information, but that information can empower people to put the kind of pressure on that might be really useful.  I think that would be a great thing for the re-authorizers to think carefully about in the next round.

Frederick M. Hess:  Checker?

Chester E. Finn:  Well, let us keep in mind that special ed. is still fundamentally about providing services, not about delivering results.  There have been some efforts at results accountability layered on top of all the service-providing mandates, requirements, rights, litigations and so on in special ed.  But it is still fundamentally not about are the kids learning; it is fundamentally about are they getting services?  NCLB is fundamentally about are the kids learning.

Michael Casserly:  It is supposed to be.

Chester E. Finn:  It is supposed to be.  We do not have a 40-year tradition in the elementary and secondary education act of machinery that focuses that way.  I think I am seeing slightly differently what everybody else up set up.  In 1965, when this law was first passed—this is the successor to the original ESEA of 1965—

Frederick M. Hess:  And when Checker was already a young man.

Chester E. Finn:  I was already an old man.  It was about getting money out for additional services to be provided to disadvantaged kids, and the logical way to do that was to give the money to the existing hierarchy - state education agencies, local education agencies because they were the ones that had the other money; and this was a perfectly logical way to get money out.  But it was not about results; it was certainly not about interventions and restructurings and corrective actions and all these other things. 

We are still using the Federalism Architecture of 1965 with the federal government assuming that state education departments can do something, which, in turn, rely on local education agencies to do something.  That was one way to hand out money, but it is not a very good way to bring about structural change in the very agencies that are in charge of the implementation of the program.

Gerry Bracey [phonetic]:  Hi, I am Jerry Bracey, independent researcher.  Most schools have 37 subgroups to report out.  If the school is working for 36 of them… let us say it is working for everybody, 36 of them are making AYP and maybe the special ed. kids are not, why should the other 36 groups have the capacity to take the choice option?  That would surely simplify that hammer.

Michael Petrilli:  I think it is a reasonable question.  When the law was going through, the concern was that if you narrowed the choice option to only kids who were doing poorly, you would create an incentive for schools to try to push those kids out.  So they would say, “Hah!  If we could get rid of those 10 kids we would make AYP.”  That was the conversation.  I do not know if those figures are founded or not but that is why the law was written as it was.  I guess I am done.

Kwesi Rollins:  Kwesi Rollins with the Institute for Educational Leadership.  I am curious about national standards and America's testing capacity.  It seems to me that… well, I would like to hear from the panel if that is really part of the solution, long-term.  It seems if you had national standards and you had the capacity to have all test results back at a reasonable time, the immediate result would be more schools not meeting AYP and kind of a huge crisis. 

It seems that part of the solution is scope and sequence of some of the concepts in the tool kit.  I wonder in a system like if you use DC for an example where currently maybe 2/3 of the schools are not meeting adequate yearly progress, you have a fairly robust choice system, at least on paper in terms of the number of charter schools.  But they are not doing well either, so parents do not have choice.  So where does national standards fit into all of that?

Frederick M. Hess:  Mike?

Michael Casserly:    I think national standards will, maybe, cure some things but it is not going to cure everything by any stretch of the imagination, and certainly not going to solve this problem about the late submission of data.  As a matter of fact, it is conceivable it could make it a little bit worse.  On the other hand, if the United States has any pretense at all about being first in the world in reading and mathematics and science and in other subjects, leaving it to 50 individual states to define what proficiency means is really nonsense. 

It is no way for the country to get there.  And it is impossible for any individual state or district to really compare themselves across state lines in ways that will help you figure out what works and what does not work.  The standards alone probably would not do it.  You would have to require the states tether their individual tests to those standards with common definitions of what proficiency are, and then you could maybe have some hope of creating some logic and some coherence around a national education strategy.

Frederick M. Hess:  I am not going to let Mike and Checker address this because it will go on forever.  All right, one quick sentence.

Michael Petrilli:  Thanks Rick.  That is what I get for… you know, he is my Podcast buddy.  Okay, so very quickly, we could imagine a system whereby there were national standards and tests, a real national accountability system that measured every school the same that determined which schools were in need of improvement; and then stop.  And the states and the local districts would decide, then, what to do about those schools that need improvement.  That might be a more workable federal role, to be the performance measurement system but not to try to get involved in the actual sanctions and remedies.

Jane Hannaway:  Jane Hannaway of the Urban Institute, and, again, I am cheating also because I’ll be speaking later.  But I’m