|Education Outlook logo 130|
No. 2, April 2008
In 1983, the seismically influential Reagan-era blue-ribbon report A Nation At Risk declared, "If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war." Twenty-five years later, when it comes to the crucial task of preparing our children for citizenship, how do we fare?
In the aftermath of the furor provoked by A Nation At Risk, the National Assessment of Educational Progress (NAEP) conducted a groundbreaking study to determine what America's seventeen-year-olds knew about history and literature. Administered in 1986, the tests revealed the disheartening answer: not enough. As Diane Ravitch and Chester E. Finn Jr. gravely concluded in their 1987 book What Do Our 17-Year-Olds Know?, "It is impossible to avoid the conclusion that something is gravely awry. . . . Our eleventh graders as a whole are ignorant of much that they should know. We cannot be certain that they were taught it; but the evidence is unmistakable that they do not know enough of it."
Unfortunately, today there is no good measure of how much our children know about American history and literature. While the NAEP evaluates twelfth-grade students' knowledge of history roughly every five years, it excludes youths who are not enrolled in school, and according to the Department of Education, only one-third of the questions test historical "knowledge and perspective," while the other two-thirds test historical "analysis and interpretation." There is no ongoing effort to assess knowledge of literature.
A new study that I authored for the Washington, D.C.-based nonprofit Common Core seeks to fill this void and determine just how much today's seventeen-year-olds know about history and literature. While the findings cannot be readily compared to those collected in 1986--given substantial differences in how the tests were administered and how the data were collected--they offer valuable insights into where we stand today. Because the data were collected using questions developed, refined, and administered as part of NAEP, they represent a carefully designed measuring stick.
For the Common Core survey, 1,200 telephone interviews were completed in the first two weeks of January 2008 using a targeted sample base of 32,000 records purchased from Scientific Telephone Samples (STS). This list was part of a nationwide STS database of over 1.6 million seventeen-year-olds. Based on the sample size, the margin of error is plus or minus 3 percent. STS administered questionnaires consisting of thirty-three multiple choice questions, including twenty-two on history and eleven on literature, all drawn from the 141 history and 121 literature questions administered in 1986.
What Our Seventeen-Year-Olds Know
In their analysis of the 1986 assessment, Ravitch and Finn proposed a grading system modeled on the scale widely used by America's classroom teachers, in which 100 percent is a perfect grade, below 60 percent is a failing mark, and letter grades are marked off in ten-point increments. In other words, 90 percent and up constitutes an A, 80-89 percent a B, 70-79 percent a C, and 60-69 percent a D.
By these standards, how did today's seventeen-year-olds fare? On the whole, students answered 67 percent of the thirty-three questions correctly, earning a cumulative grade of D. On the history section, they earned a C, answering 73 percent of questions correctly. When it came to literature, they earned an F, correctly answering only 57 percent of the questions.
More disturbing than these aggregate results may be some of the items that many seventeen-year-olds did not know. Nearly a third could not identify "ask not what your country can do for you" as the words of President John F. Kennedy. The same fraction did not know that the Bill of Rights enshrines our rights to freedom of religion and speech. Two in five could place the Civil War in the correct fifty-year period, and half knew that the Federalist Papers were written to encourage ratification of the U.S. Constitution. Nearly a quarter could not identify Adolf Hitler. Less than half could identify the literary figures of Job or Oedipus, while barely one in two could identify the plot of George Orwell's immortal 1984.
History. Table 1 presents the weighted history results. The questions assessed students' knowledge of U.S. and world history, with five questions on U.S. presidents, four on other historic individuals, three about the dates of major historic events, two about the design of the U.S. government, and the other eight on topics such as the European Renaissance and the U.S. Supreme Court's 1954 Brown v. Board of Education ruling. By almost any measure, the questions asked were basic.
How well did seventeen-year-olds fare collectively on the twenty-two history questions? There was one question on which they earned an A (with at least 90 percent correct) and only five more on which they earned a B (with 80-89 percent answering correctly). There were ten questions on which respondents earned a C (with 70-79 percent answering correctly) and two on which they earned a D (with 60-69 percent answering correctly). There were four questions that fewer than 60 percent of students answered correctly.
Note the five history questions on which students fared worst. One asked respondents to identify the European Renaissance, and another inquired about the intentions of the authors of the Federalist Papers. Sixty-one percent of seventeen-year-olds correctly identified the Renaissance as a period of "technological and cultural advances." In reference to the Federalist Papers, half of the respondents knew that they were intended to promote ratification of the U.S. Constitution. Two of the questions on which students performed most poorly asked respondents to identify the approximate period in which historic events took place. Fewer than 60 percent could identify the correct era in which World War I occurred, and only 42.6 percent knew the Civil War was fought between 1850 and 1900. On another topic, barely half of the respondents were aware that the controversy surrounding Joseph McCarthy focused on communism.
Literature. Table 2 presents the weighted literature results. The questions assessed students' knowledge of both contemporary and classic works, with six questions coming from novels, two asking about poems, two inquiring about plays, and one referencing the Bible. Once again, note that the questions asked are, by nearly any measure, basic.
How well did seventeen-year-olds fare collectively on the eleven literature questions? As a whole, they earned three Cs, one D, and seven Fs. There were only four questions that even 60 percent of respondents answered correctly. Students failed seven of the eleven questions. Certainly, skeptics might suggest that literature knowledge would be better measured by standards drawn from more recent works. But the purpose of this survey was to measure seventeen-year-olds' knowledge of their literary heritage, not their exposure to popular culture.
Note the five literature questions on which survey respondents fared worst. Regarding George Orwell's 1984, 52 percent of students recognized it as a novel about "a dictatorship in which every citizen was watched in order to stamp out all individuality." Only 41.3 percent knew that Ralph Ellison's novel about a young man growing up in the South and then moving to Harlem was Invisible Man. When asked about the author of The Canterbury Tales, 38.3 percent of those surveyed named Geoffrey Chaucer. Only 44.8 percent knew that Oedipus is the character in an ancient Greek play who "unknowingly killed his father and married his mother." Finally, 49.7 percent of respondents identified Job as being known in the Bible for his patience in the face of suffering.
A Half-Empty Glass. The problems that the above results pose for civic discourse are neither murky nor obscure. One need not search far to find attacks on antiterrorism measures that draw upon imagery from 1984 or use the term "Orwellian." Pundits, novelists, and journalists routinely wield references to Job and Oedipus when writing about the trials of a public figure or the complexities of familial relationships. High school graduates unacquainted with these terms are handicapped when it comes to engaging in such public debates, perhaps recognizing the terms and phrases but lacking comprehension of the assumptions and associations that lend them meaning. What is worse is that they have been deprived of the knowledge and wisdom that historical information and artistic works provide.
In summarizing the results of the 1986 study, Ravitch and Finn concluded that "the glass is almost half empty. . . . We cannot tell from a 'snapshot' assessment of this kind whether today's students know more or less about history and literature than their predecessors of ten, twenty, or fifty years ago. We do conclude, however, that they do not know enough." More than twenty years later, it is safe to say that the story remains equally disheartening.
The Impact of Parental Education
History and literature can be absorbed in the home as well as at school. Students born into educated or affluent homes where books and cultural experiences abound may have the opportunity to become culturally literate regardless of what happens in their K-12 schooling. But for those without such advantages, school offers their only chance of acquiring this necessary knowledge. Comparing the results of seventeen-year-olds with at least one college-educated parent to those without reveals that students from more educated families have a sizable edge in their knowledge of history and literature.
Seventeen-year-olds born into more educated environments are approaching high school graduation with significantly more literary knowledge than their counterparts.
In history, seventeen-year-olds with a college-educated parent scored at least one full letter grade above those without a college-educated parent on over 40 percent of the history questions. They earned a D or an F on three questions, whereas those without college-educated parents earned three times as many. Those less-advantaged respondents earned an A or B on four questions, while respondents with college-educated parents earned twelve. The biggest difference between the two groups emerged on the questions that asked about the First World War, the Renaissance, and the ethnic population that America interned during World War II. In each case, the difference between the two groups was between fifteen and twenty percentage points--nearly two letter grades. The smallest differences emerged when respondents were asked about the time period of the Civil War and the bombing of Pearl Harbor.
In literature, even seventeen-year-olds with a parent who had graduated from college managed a passing grade on only four of the eleven questions, while earning an F on seven. The biggest differences between students of college-educated and non-college-educated parents emerged on the questions that asked them to identify Uncle Tom's Cabin, Oedipus, and Geoffrey Chaucer. About poor Oedipus, respondents with more educated parents fared twenty-one percentage points better. On the other two questions, the differences between the two groups were between ten and fifteen percentage points.
While both groups fared poorly on the history and literature questions overall, seventeen-year-olds born into more educated environments are approaching high school graduation with significantly more literary knowledge than their counterparts. While aggregate performance is mediocre, those teenagers with a parent who graduated from college did substantially better than their peers. In a nation concerned about social divisions and civic apathy, this is a worrisome state of affairs and one deserving careful attention. One way to interpret these results is to consider what respondents had to say about the extent of their participation in various cultural opportunities like attending plays, visiting museums, singing in choirs, and reading at school and at home.
Approximately two-thirds of all seventeen-year-olds have attended a play or read a work of literature outside of school, and about half have visited an art museum or participated in a choir or orchestra. About three-quarters of those with a college-educated parent have attended a play and have read at least one work of literature outside of school, while about half have visited an art museum or participated in a choir or orchestra. Among those young people without a college-educated parent, the figures are markedly lower. These teenagers were sixteen percentage points less likely to have read at least one work of literature outside of school and twelve percentage points less likely to have attended an art museum. They were also eight to nine percentage points less likely to have attended a play or participated in a choir or orchestra.
There is reason to believe that the substantial gap in knowledge is due to the prevalence of books and literary material in the home, parental interests and activities, and the assignments given at the schools that they attend. Any effort to untangle these various threads, however, and to determine their impact or relative importance is beyond the scope of this analysis. What is clear is that all students need improved instruction in literature and history--and that less-advantaged students would benefit most of all.
When it comes to familiarity with major historical events and significant literary accomplishments, America's seventeen-year-olds fare rather poorly. When asked relatively simple multiple choice questions and graded on a generous scale, teens on the cusp of adulthood earn a D overall. Moreover, there is an unsurprising but unfortunate gap between those children born into homes headed by college-educated parents and their peers. When it comes to familiarity with the base of knowledge that enables us to engage in conversations about values and policy, our seventeen-year-olds are barely literate.
When it comes to familiarity with the base of knowledge that enables us to engage in conversations about values and policy, our seventeen-year-olds are barely literate.
Perhaps these results should not come as a great surprise. For all the attention paid to school improvement in recent years, particularly at the high school level, the focus has been on reading, math, and graduation. It appears likely that this focus has not served the broader aim of ensuring that our children are educated in the liberal arts and sciences.
What should we take from these findings? For starters, it is essential that parents, educators, and policymakers examine how to approach the teaching of history and culture. We must ask whether popular reform currents are delivering the results (and incentives) we wish and what that means for school reform going forward. Five specific recommendations deserve attention.
First, we need to assess more systematically student learning beyond math and reading, particularly in the subjects of the traditional liberal arts. This does not mean adding new assessments into the No Child Left Behind (NCLB) framework--in truth, it is probably advisable not to do so--but it does suggest that states, school districts, foundations, and the National Assessment Governing Board should think hard about how we might better measure learning in the liberal arts at a variety of grade levels.
Second, as we debate the reauthorization of NCLB and design state accountability systems, we would be well-advised to reassess whether these systems or district practices are promoting or unintentionally stifling instruction in the liberal arts and sciences. The Common Core survey makes clear that students' knowledge in these areas falls far short of where it needs to be. In light of extensive efforts to promote educational accountability in recent years, it is worth ensuring that policies such as NCLB are helping to address this challenge and not somehow aggravating it.
Third, we must do a substantially better job of teaching the liberal arts. In the current policy environment, a majority of time and energy is being devoted to research about reading and math instruction. This work is essential and invaluable, but it would behoove policymakers and educators to ensure that their enthusiasm for basic skills is not marginalizing attention to questions of civic importance. To this end, it may be useful to consider the Core Knowledge, Latin, or International Baccalaureate programs, which could offer promising approaches and useful lessons.
Fourth, we need to make sure that our teachers have the knowledge they need to teach the liberal arts and sciences well. Unfortunately, there is little energy or attention devoted to programs that ensure teachers are equipped to provide such instruction. There is a need to research the state of teacher mastery and to devise recruitment and training programs that will promote robust teaching and learning in the areas of historical and cultural knowledge.
And finally, it is important to note that encouraging a rich liberal arts and sciences education for all is an idea around which reform advocates of various stripes can rally. Whatever reforms one believes can promote quality schooling, we can all agree that "quality" includes a broad, rich, and challenging liberal arts curriculum. In truth, proponents of charter schools or vouchers, mayoral control or school boards, merit pay or career ladders, and any number of other measures all have reason to insist that these structural arrangements be coupled with a rich curriculum provided equitably to all students. A successful coalition on behalf of liberal education can and must welcome those who may otherwise disagree on the particular shape of reform.
In profound and essential ways, our civic health and national cohesion depend on our ability to familiarize the rising generation with the touchstones of our shared history and culture. Ensuring that all citizens have a shared sense of Martin Luther King Jr.'s "I have a dream" speech and the attack on Pearl Harbor are a start--but only a very modest one. Alone, such scattered kernels of awareness constitute no more than a handful of romanticized images flickering in the national conscience. What we need is confidence that all of our children will be familiar with the highs and the lows of the compelling narrative that is our common heritage--because a nation without this is truly a nation at risk.
Frederick M. Hess ([email protected]) is a resident scholar and the director of education policy studies at AEI. A copy of the Common Core report Still at Risk: What Students Don't Know, Even Now is available online at www.aei.org/publication27576/.
AEI researchers Thomas Gift, Rosemary Kendrick, and Juliet Squire and web editor Laura Drinkwine worked with Mr. Hess to edit and produce this Education Outlook.
1. National Commission on Excellence in Education, A Nation At Risk: The Imperative For Educational Reform (Washington, DC: U.S. Government Printing Office, 1983), 5.
2. For the official report, see Arthur N. Apple, Judith A. Langer, and Ina V. S. Mullis, Literature & U.S. History: The Instructional Experience and Factual Knowledge of High School Juniors (Princeton, NJ: Educational Testing Service, 1987).
3. Diane Ravitch and Chester E. Finn Jr., What Do Our 17-Year-Olds Know? A Report on the First National Assessment of History and Literature (New York: Harper & Row, 1987), 200.
4. For instance, a question that tests no precise historical context provides a table listing household expenditures on various appliances in 1900 and in 1928. Appliances purchased in 1900 include bicycles, a wringer and washboard, brushes and brooms, and a mechanical sewing machine. Those purchased in 1928 include an automobile, radio, washing machine, and vacuum cleaner. The question asks: "Create a thesis statement about the way the changes shown in the table above affected everyday life between 1900 and 1928," and "use the information in the table to help prove your thesis." A second example of this type of question is one about a cartoon that depicts Lyndon B. Johnson clinging to the side of a gear labeled "Vietnam War" as it rotates toward a gear labeled "1968 election." The question posed is: "What is the main message of this cartoon about Lyndon B. Johnson?"
5. Detailed descriptions of the criteria governing the selection of the initial items for literature and history can be found in National Assessment of Educational Progress (NAEP), Foundations of Literacy: A Description of the Assessment of Basic Knowledge of United States History and Literature, NAEP Description Booklet no. 17-HL-11 (Princeton, NJ: Educational Testing Service, 1986).
6. More information about Scientific Telephone Samples can be found on its website, www.stssamples.com.
7. Diane Ravitch and Chester E. Finn Jr., What Do Our 17-Year-Olds Know? A Report on the First National Assessment of History and Literature, 45.
8. Ibid., 120.