|Education Outlook logo 130|
No. 8, August 2009
NCLB requires states to establish performance benchmarks and test students in reading and math in grades three through eight, and at least once in high school, to ensure that requisite percentages of students at each school are meet-ing the reading and math targets. If an insufficient percentage of students in any of a school's subgroups (identified by race as well as by socio-economic, special education, and English-language-learner status) score "proficient" on the state assessment in reading or math, the school is labeled as failing to make AYP. Schools must be "restructured" by the district if they fail to make AYP for five consecutive years.
NCLB lists five restructuring options: "entering into a contract to have an outside organization with a record of effectiveness operate the school; reopening the school as a charter school; replacing all or most of the school staff who are relevant to the failure to make AYP; turning operation of the school over to the state, if the state agrees; or undertaking any other major restructuring of the school's governance."
Hawaii's island status has made it difficult to secure the talent and expertise it needs to address the problems of failing schools, necessitating a reliance on contracting with external providers. As the only state in the nation with a statewide school district that operates as both a "state" and "local education agency," Hawaii also has substantial freedom in designing intervention strategies and in forging partnerships with external providers.
To gauge the impact of Hawaii's restructuring efforts, we compared the gains made by restructuring schools under contract with external providers relative to the gains made by restructuring schools not under contract with external providers. Schools not using comprehensive providers are restructured by regional superintendents, which Hawaii deems "complex area superintendents" (CAS). We compared the average increases in the percentage of students deemed "proficient" on the Hawaii Content and Performance Standards over a three-year period, beginning in 2005, when Hawaii's restructuring strategy was initiated. In 2005, thirteen elementary and middle schools were planning for restructuring, and forty were implementing restructuring. Seventeen worked with an external comprehensive provider, and the remaining thirty-six were restructured by a CAS. We also compared the fifty-three schools that entered restructuring in 2006 over a two-year period. In 2006, three schools were planning for restructuring, and forty-seven schools were implementing restructuring. Thirty of the fifty-three schools partnered with an outside provider.
If Hawaii's restructuring strategy is successful, then restructuring schools that partner with an external provider will make greater achievement gains than restructuring schools that choose not to partner with an external provider.
In Figures 1-4, we see two clear patterns. First, of the schools that entered restructuring in 2005--the first year of Hawaii's restructuring strategy--those schools without external partners appeared to perform as well as those with external partners. Schools without external providers outperformed those with external providers in mathematics (both in grades three through five and grades six through eight) and in reading (grades three through five). Schools with external providers made substantially greater gains than schools without external providers in reading in grades six through eight. However, after the first year of using external providers--for the fifty-three schools in restructuring from 2006 to 2008--the story is quite different. Schools with external partners made greater gains than schools without external providers in mathematics (both in grades three through five and six through eight) and reading (grades six through eight). They also gained slightly more ground in reading in grades three through five. Whether these data reflect a first-year adjustment period under a new strategy or are an indication of the effectiveness of the diverse provider model, they are cause for further study.
Given the short time frame, the results are inconclusive. The reason to examine Hawaii is not necessarily to emulate its current approach, however, but to glean insights from its particular experiences and understand whether and how its approach might be employed to more substantial effect.
The Hawaii Context
Hawaii's school enrollment has decreased from a peak of 189,000 students in 1997-98 to 178,000 students in 289 schools today. Of these students, one in four are Hawaiian or part-Hawaiian, one in five are Filipino, and one in ten are Japanese. Just 15 percent are Caucasian. The total budget for Hawaii public schools is approximately $2 billion, with per-pupil expenditures hovering slightly above the national average. On the 2007 National Assessment of Education Progress, 37 percent of Hawaii's fourth-grade students scored "proficient" or above in mathematics, and 31 percent scored "proficient" or above in reading. Just 24 percent of eighth-grade students scored "proficient" or above in mathematics, and just 21 percent did so in reading.
State superintendent Patricia Hamamoto has worked in Hawaii's schools for nearly thirty years. She notes that a small hiring pool in Hawaii makes it difficult to find new principals because over 50 percent of new hires come from out of state: "We don't have the depth" to replace principals. Efforts to enhance school leadership are necessarily focused on improving people already in place rather than recruiting replacements. Wendy Takahashi, the principal of Oahu's Nanakuli Elementary School, notes the same challenge with teachers: "We do all of our training here, and then they leave to go closer to home." A January 2008 report from the Hawaii Educational Policy Center reports the annual need for fully licensed teachers (including attrition and unlicensed teachers) at 30 percent of the state's teaching workforce--nearly double the national teacher turnover rate.
Hawaii has historically lagged in matters of assessment, standards, and curricular coherence. Denise Matsumoto, who served on the board of education from 1988 to 2008, recalls that the state first adopted standards in 1994 and did not have an accountability system prior to the enactment of NCLB, leading to a dramatic degree of fragmentation in curriculum, assessment, and instruction. This context makes the state's role in providing data and transparency a central element of its restructuring initiative, but it suggests that these efforts reflect a state playing catch-up to its neighbors rather than breaking new ground.
The replacement of school staff and the use of charter conversions are absent thus far in Hawaii's restructuring. Sharon Nakagawa, the administrator of the state's Special Programs Management Section (SPMS), which oversees restructuring, says that "NCLB [makes clear that it does] not supersede state laws, MOUs [memoranda of understanding], or union agreements. That limits what Hawaii can do in terms of removal of people." While charter conversion is an option for restructuring schools, no schools thus far have taken that course--which could be attributed to union opposition and reduced per-pupil funding. Matsumoto notes that the "guessing game" on funding has made it difficult for charters to know what to expect.
Hawaii's Approach to NCLB Restructuring
In 2003, the Hawaii Department of Education (HIDOE) developed a restructuring framework, which the board approved. The framework is frequently a personal strategy that involves informal collaboration rather than strict procedures or processes, and, while the board of education writes policies, implementation is largely left to the discretion of the superintendent. The framework relies heavily on comprehensive providers' partnerships with schools and accords a key role to the CASs.
HIDOE officials saw a profound need for external expertise to restructure the state's failing schools, but Nakagawa recalls, "We knew that there was tremendous opposition to a managed system." Rather than hand over operational authority to an external organization, HIDOE's SPMS envisioned a partnership model in which school leadership collaborated with comprehensive providers.
With this in mind, the SPMS issued a request for proposals (RFP) in the fall of 2004 and received ten applications. An initial five-person committee from the SPMS vetted the proposals for key elements: curricular and instructional support, staff and leadership development, school culture, staff accessibility, and a built-in assessment system accompanied by the expertise to assist teachers with data analysis. Three of the ten applicants met the state's criteria: America's Choice, EdisonLearning, and ETS. (America's Choice, which has worked in Hawaii since the 1990s, lacked the built-in assessment system but was approved when it subcontracted with SchoolNet.) These providers began working in Hawaii's restructuring schools in the fall of 2005. A similar RFP for comprehensive providers is issued annually and is now reviewed by a team of twenty-five individuals from the school, complex, and state level.
The strategy delegates a crucial role to CASs, who select the restructuring strategy for schools in their complex area. According to the framework, each CAS "determines which restructuring option will be used; directs and manages the school's restructuring efforts; makes leadership decisions for the school, including personnel decisions; manages all curriculum and instruction for the school; and exercises budgetary authority over all school funds and resources, except funds designated to the restructuring provider." Each year, HIDOE has invited CASs to an annual statewide "fair" with the comprehensive providers, allowing them to select which, if any, provider will work with their restructuring schools. Patricia Park, CAS for the Leilehua-Mililani-Waialua complex area, explains, "You could go to different sessions and hear the different sales pitch from all of them." While budget limitations prevented HIDOE from hosting the fair in 2008, it distributed an extensive brochure listing the providers and describing the services offered.
CASs function as arms of the superintendent rather than as independent operators and actually earn less than principals, with a pay scale that tops out at more than $10,000 below the scale negotiated for principals by the influential Hawaii Government Employees Association. CASs encourage differing levels of input from school leaders. Lisa Joy Andres, a school renewal specialist for the Honolulu schools, notes that in some cases, "The CAS left it up to the principals which [provi-der] to consult with," while Art Souza, superintendent of the West Hawaii complex area on the Big Island, relates, "The ultimate decision was with me, and I had to make sure that that was understood by everyone."
The SPMS of HIDOE has streamlined the procurement process for America's Choice, EdisonLearning, and ETS, drafting statewide contracts and getting the requisite approval of the state attorney general. Previously, school principals who partnered with outside firms had worked with the central procurement office to write individual contracts, but under the new framework, school officials simply need to fill out an invoice. Elaine Takenaka, a consultant to the SPMS who helped to develop the contracts, notes that this new approach allows providers to work with a school with a lead-time of several weeks compared to several months.
Takenaka explains that, between 2005 and 2008, Hawaii was able to fund the costs of restructuring with a focus on helping schools make AYP. Fiscal educational specialist Iris Mizuguchi conveys that, in light of the economic downturn and looming budget cuts, "this coming school year is going to be a big challenge." Dane Linn of the National Governors Association has suggested that a statewide contract as opposed to a CAS-made decision could yield a more competitive price for the providers, but budget limitations in the current fiscal environment promise tough decisions ahead.
Making Hawaii's Approach Work
Five elements help explain the praxis of Hawaii's restructuring. These elements suggest that the state's unique use of external partnerships is accompanied by efforts to lend political cover, build infrastructure, and provide crucial support and monitoring.
Building Relationships and the "Coconut Wireless." Hawaii's restructuring effort is informed by the state's close-knit culture, which plays a critical role in sharing information, helps school leaders and CASs to be informed consumers of external assistance, and requires that effective providers tend to their professional relationships as well as technical acumen. Nina Buchanan of the University of Hawaii, Hilo, emphasizes the importance of building relationships: "It's not necessarily about the model itself but about the personal nature of who is delivering it and whether they're available to ask questions and to help."
Park agrees: "In the end, a lot depends on relationships and expertise and knowledge. So you've got to do the relationship first and then the people are willing to accept the knowledge and the expertise and the assistance." Practitioners and administrators consistently cited the expertise and accessibility of the providers as critical factors in a school's ability to make progress.
Nakagawa notes that when schools begin to see improvement, news spreads by "Coconut Wireless," an informal word-of-mouth communications system that rapidly disseminates information between schools. "When [principals] begin to see how they're improving and how they're becoming more successful, that's very empowering. . . . [Then] other principals entering restructuring see the significant changes and improvement of other schools and will request these providers. . . . Success breeds success." For this reason, providers have cause to take their reputations and relationships very seriously.
A Data-Rich Environment. A priority for superintendent Hamamoto has been promoting performance transparency and ensuring the availability of data. The most significant effort on this front is the requirement that schools conduct quarterly assessments, in addition to the state's annual assessment. Hamamoto states that Hawaii is currently in the process of reducing the number of choices, as schools are not required to use any one assessment system. Restructuring schools typically use those provided by their partner organization.
While the multiplicity of assessments has led to fragmentation and prevents comparison across schools, it allows schools to shape instruction based on more frequent and systematic data. A teacher at Palolo Elementary School stated, "[We] get a printout of the assessment and see which items a student missed--and then we can reteach those items." King Intermediate School principal Sheena Alaiasa says, "It's all data--everything is data-driven." This focus on data is relatively new to Hawaii. State board chair and former state senator Donna Ikeda observes, "It's a real change. When I was in the Senate, getting info . . . was like pulling teeth."
Hawaii has also implemented a "comprehensive needs assessment"--essentially a "forensic" audit of schools entering restructuring. Hamamoto explains that it entails a two- to three-day assessment of personnel, practices, governance, data systems, and instruction at the school in question. This report becomes the basis for diagnosis and determining intervention strategies. While Ikeda laments that the assessment amounts to less of a "hammer" than a counseling session, Souza notes that the needs assessment provides the opportunity to "formulate a plan that will determine what level of restructuring and what kinds of interventions are needed." Various CASs, providers, and principals have credited the needs assessment with reinforcing the state's emphasis on data and accountability, creating transparency, and providing an opening for uncomfortable measures.
Political Cover. Perhaps the most commonly echoed refrain from people in various education positions is the degree to which island culture emphasizes harmony and personal relationships. John Kreick, EdisonLearning's regional general manager in Hawaii, said, "You don't make waves. This is not a culture where you call people out publicly." Consequently, principals have historically balked when faced with making unpopular decisions, promoting individual accountability, or upending familiar routines. But as CAS Lisa Delong states, "Restructuring gives everyone a little more authority." Souza explains, "One of the key roles the external providers play is that they can be the bad cop when they have to."
Linn agrees that "anytime you bring in an external provider, it takes the monkey off the principal's back." However, he suggests that limiting the authority of external providers by bringing them in as partners, rather than as management, has diminished their impact. "They've gone one step further, but they haven't gone all the way. . . . Let the people who know how to do this make the decisions."
Streamlining Procurement and Providing Contract Support. As CASs have sought to identify and work with comprehensive providers, an important ingredient has been HIDOE's efforts to shield CASs and schools from distractions and obstacles. CASs and principals did not have to deal with union opposition because it was addressed at the state level. Andres, explaining how political complications are addressed, says: "The state does all of that for us. . . . By the time it comes to our level, my complex area superintendent says, ‘Here are three [providers]. This is who we have to choose from.'" Linn recognizes the benefits that it can offer: "In other places . . . the responsibility is--legally and logistically--the district's. . . . There is no interaction between district and state on any procurement issues. . . . I bet districts would love to have that streamlined."
More broadly, HIDOE has sought to craft a manageable and user-friendly procurement process. Under the new system, Andres explains, "We know what they're providing and all of the details that are pertinent to the schools. So when [a CAS] decides to hire them, they can just write it into their budget proposal and academic and financial plans for the following year, knowing that the provider has already been approved by the procurement office." By taking responsibility for writing contracts with providers and negotiating the approval process, HIDOE has made straightforward a process that was once drawn out and painful.
Quality Control and Monitoring. A persistent challenge for Hawaii has been policing the quality of potential providers, discouraging schools from making bad choices, and ensuring accountability for both schools and providers. Through the procurement process, HIDOE reinforces the focus on data and accountability by employing an evidence-based standard in determining which providers to approve and requiring that approved providers have a developed data system that can be installed in the schools.
Perhaps Hawaii's most important steps on accountability are its quarterly student assessment requirements and the creation of the "significant improvement" metric for schools. That data is used by CASs and state officials to monitor performance, push school leaders to assess providers' performance, and ensure that all contracting decisions are focused on outcomes. In addition, providers meet quarterly with principals, CASs, and the state SPMS office to review their progress. As Elden Esmeralda, principal of Nanaikapono Elementary School, says, "At some point, maybe, if . . . the gains aren't quick enough, [the CAS] may say we can't" continue with current providers. Nakagawa adds, "The CAS or the superintendent may determine that a school's student performance is flat-lining and may require the school to select a different model of services or a different provider."
This attention to results is rare on the mainland, where, Linn observes, "[v]ery few states have any level of accountability for providers. . . . They feel as though once they turn over the contract to an independent provider, they've fulfilled their responsibility." Gene Wilhoit, executive director of the Council for Chief State School Officers, suggests that Hawaii's unique structure may allow "a cleaner mechanism and you might get much more direct control over the process where you have that kind of reporting mechanism going directly to the state level." In perhaps the clearest illustration of HIDOE taking an active role in quality control, Hamamoto reports, "We've dropped America's Choice from several schools and have gone with EdisonLearning and others" because the data showed those America's Choice schools were making insufficient gains.
No one should mistake Hawaii's restructuring model for a wholesale effort to rethink or reinvent Hawaii's restructuring schools. The Hawaii approach does not entail changing staff, reforming governance, redesigning schools, or turning school operation over to the state. Rather, it is an attempt to apply focus, data, and expertise in a challenging state environment by encouraging the use of diverse providers. However, Hawaii's experiences highlight a set of takeaways for how state and federal policymakers might facilitate an effective diverse provider model.
First, the trust and communication within Hawaii's K-12 community plays a crucial role in disseminating information on the quality of different comprehensive providers and encourages providers to take their relationships and reputations seriously. Second, saturating the district with a focus on data-supported results has helped the state hold providers accountable. Third, the presence of comprehensive providers can serve to strengthen the hand of school leadership to make difficult decisions. Fourth, streamlined procurement means that providers' employment is relatively frictionless, demands on school personnel are minimized, and HIDOE is able to monitor the quality of the contracted providers.
Four questions loom for Hawaii and those seeking to learn from Hawaii's experience. First, will Hawaii's collaborative, "soft-edged" restructuring model prove equal to the challenge of boosting student achievement dramatically in persistently low-performing schools, or will this model need to be made more aggressive? Second, will providers, who have seen gains in a limited number of locations, enjoy equal progress as they begin to serve more schools? Third, how can Hawaii's experience inform restructuring in other states that rely more on formal processes or have more decentralized systems of governance? Finally, given the lack of research on restructuring strategies across the nation, how well are various other approaches faring? With the reauthorization of NCLB looming, these questions are ripe for further investigation.
Frederick M. Hess ([email protected]) is a resident scholar and the director of education policy studies at AEI. Juliet P. Squire ([email protected]) is the program manager in education policy studies at AEI.
1. Frederick M. Hess and Michael J. Petrilli, No Child Left Behind: A Primer (New York: Peter Lange, 2006), 53.
2. Ibid., 1.
3. This analysis draws on information from a variety of sources, including documents, newspaper reports, materials supplied by providers and the Hawaii Department of Education, publicly available data, and more than three dozen interviews and conversations with educators, service providers, and policymakers. The authors are particularly grateful for the assistance of state superintendent Patricia Hamamoto, two current or former chairs of the state board of education, Hawaii's Title I officials, the head of Hawaii's teachers union, a number of complex area superintendents, a half-dozen principals, comprehensive service providers, and more than a dozen school faculty and school support personnel. The authors would also like to thank EdisonLearning's Tung Le and David Yang for their assistance in compiling Hawaii's student achievement data.
4. Frederick M. Hess and Michael J. Petrilli, No Child Left Behind: A Primer, 31-32.
5. Ibid., 53.
6. Caitlin Scott, A Call to Restructure Restructuring: Lessons from the No Child Left Behind Act in Five States (Washington, DC: Center on Education Policy, September 2008), 10, available through www.cep-dc.org (accessed August 6, 2009).
7. Hawaii Department of Education, "Supplementary Data Tables," 2007, 6, available at http://arch.k12.hi.us/PDFs/state/superintendent_report/2007/AppendixDataTables07rptforPDFr1.pdf (accessed August 6, 2009).
8. Patricia Hamamoto, "Superintendent's 18th Annual Report," Hawaii Department of Education, 2007, 15, available at http://arch.k12.hi.us/PDFs/state/superintendent_report/2007/2007SuptRptFinal20080320.pdf (accessed December 16, 2008).
9. Hawaii Educational Policy Center, "Teacher Education Workforce Research Group: Report to the Legislature," January 2008, available at www.hawaii.edu/hepc/pdf/Reports/SCR_56 _HEPC_Interim_Report_0108.pdf (accessed August 6, 2009); and Raegen Miller and Robin Chait, Teacher Turnover, Tenure Policies, and the Distribution of Teacher Quality (Washington, DC: Center for American Progress, December 2008), 10, available at www.americanprogress.org/issues/2008/12/pdf/teacher_attrition.pdf (accessed August 19, 2009).
10. Informed sources differ on this count, with some suggesting that no schools have used this strategy and others that two schools have.
11. Hawaii Department of Education, "Draft Framework for School Improvement: Overview," Office of Curriculum, Instruction, and Student Support, June 25, 2008, 13-1.