Dimensions: 5.5'' x 8.5'' 140 pages AEI Press  (Washington) Publication Date: February 1999 Paperback ISBN: 0844741027 Price: $ 16.95 Add to Cart   Examination Copies

The decennial census mandated by the Constitution is a matter of great importance. The census provides the basis for establishing political boundaries, including boundaries for state and local political districts as well as congressional districts. It also serves as a basis for fund allocation: not only federal funds but also state, local, and private funds are often distributed on the basis of census data. The census enables scholars, government officials, business people, planners, and citizens to understand trends and developments in individual communities as well as in the nation as a whole.

This volume argues against the controversial proposal to use sampling techniques in the next federal census. Sampling would be used to estimate the size of segments of the population that many researchers believe to be undercounted by traditional census methods.

Kenneth Darga is senior demographer at the Michigan Department of Management and Budget.

Most popular accounts of the controversy over the use of census sampling suggest that all relevant scientific arguments favor the plan of the Census Bureau to adjust the 2000 Census for an undercount. Supporters of that proposed approach often imply that any opposition to it must be based on brazen partisanship or on resistance to scientific methods.

Although that view is pervasive, this book demonstrates that it is not accurate; there are compelling scientific arguments against the proposed approach. The book does not argue against sampling in the abstract; it argues instead against the particular approach to adjustment based on the sampling method proposed. Even an otherwise valid statistical technique tends to produce faulty results when it relies on faulty data and faulty assumptions.

The Problem and the Proposed Solution

It is hard to conduct a census. Despite the best efforts of the census takers, some people will be missed and some people will be counted in error. The 1990 Census, for example, appeared to have a net undercount of approximately 1.8 percent, or slightly fewer than 5 million people.

Adjustments for undercount are based on a survey that counts people again in a sample of neighborhoods. The key to measuring undercount with such a survey lies in matching the survey information with census forms. When the survey includes people whose census forms cannot be found, it is assumed that they were missed by the census. When survey forms cannot be found for people who were counted by the census, they are investigated to determine whether they really lived where they were counted by the census.

That process results in adjustment factors that are intended to reflect the net undercount for each segment of the population. Some segments of the population may be increased by 1 percent or 2 percent, some may be increased by 10 percent or 20 percent, and some may be decreased.

Although post-census surveys have been used to study undercount after every census since 1940, the results have never been used to adjust the official population counts. The proposed plan for Census 2000 would apply adjustments to every population figure in the census--not only to state and national population counts but also to population data for individual communities and individual blocks.

The Importance of Accuracy

The proposal to adjust the census for undercount has received considerable attention because of its political, economic, and legal implications. Through their effect on reapportionment and redistricting, the adjustments could play a small but important role in determining the partisan balance of the House of Representatives in the next decade. They could also influence the distribution of billions of dollars of public funds each year. Moreover, because the census is mandated by the Constitution and because it has been the subject of legislation and litigation, the proposed adjustments have important legal aspects as well.

Overshadowing all these issues, however, is the question of whether the proposed method would succeed in making the census more accurate. Any political, economic, or legal arguments for adjusting the census lose their credibility if the adjustments make the census less reliable.

A decline in accuracy would be very serious because of the many important uses of census data. In addition to being used for drawing political boundaries and distributing public funds, the census is a primary source of information on social and economic trends. Census data tell us not only about the nation as a whole but also about individual regions, states, cities, and neighborhoods. Census data are used for planning and analysis in such diverse fields as health, transportation, economic development, agriculture, and environmental quality; few public or private organizations do not make use of data derived from the census.

The adjustment factors--which are usually small, but sometimes reach 20 percent or more--would introduce new and unpredictable errors into all census data. Because of their unpredictable size and distribution, these errors would invalidate comparisons between areas and between segments of the population. Even more important, they would invalidate comparisons between different times. Some of the major differences from one census to another would merely reflect meaningless variations in error levels, but no one would know which differences were valid and which were not. Fundamental information about social and economic trends in our neighborhoods and our nation would thus become misleading and unreliable.

Fundamental Flaws of the Proposed Approach

The problem with the proposed approach is that the survey to measure undercount is even harder to implement than the census. The survey has two fundamental shortcomings that make it unsuitable as a basis for adjusting the census. Not only does it miss many of the same people who are missed by the census, but it also makes it look as though other people were missed by the census when in fact they were not.

Failure to Measure Undercount. The book discusses two components of the undercount that are particularly difficult to measure with a survey: people who are missed because they want to be missed, and people who are missed because they are homeless.

Categories of people who are less than enthusiastic about revealing their whereabouts to the government include drug dealers, illegal immigrants, people who are wanted by the police, people who think they might be wanted by the police sometime in the future, people who are behind in their child support payments, people who are dodging bill collectors, and people who just do not trust the government. Those who were unwilling to be counted by the census are not likely to step forward a few months later when their neighborhood is singled out for a special visit by someone asking more questions for the government. While the census is a well-recognized effort that targets the entire population, people who distrust the government may feel that the survey is focusing specifically on them. They may therefore try even harder to avoid the survey.

Homeless people pose an even greater problem. If the survey tried to count homeless people in July, it would not necessarily find them in the same location where they may have been counted in April. The survey therefore does not even attempt to count homeless people.

False Identification of Undercount. Given the obvious inability of the survey to reach many of the people who are missed by the census, its results in 1990 were surprising. The proposed adjustments to the census actually reflected a higher rate of apparent net undercount than the Census Bureau’s more reliable "demographic analysis" method. Even after some of the problems that had been found were corrected the following year, the total net undercount derived from the survey was very close to the level suggested by demographic analysis. How can a post-census survey produce such results if it misses many of the people who are missed by the census?

The explanation for this paradox is that people can be falsely identified as missed by the census when an error is made in matching their survey response with their census response. The Census Bureau usually figures out which person is which, but it has to contend with clerical errors, language barriers, aliases, incorrect survey information from neighbors of people who are not at home, people who move after the census, people who deceive the interviewer, interviewers who record phony data on hot or rainy days, and a host of other problems. Such problems make it impossible to achieve very high accuracy in matching the survey to the census. Unfortunately, because of an important but frequently overlooked statistical phenomenon, virtually perfect accuracy is required to make the adjustment method work.

A Statistical Barrier to Accurate Measurement

The seriousness of errors in measuring undercount is multiplied by a simple statistical phenomenon that makes it very difficult to make accurate measurements of small segments of the population. Even small rates of error in classifying the 96 percent or 98 percent of the population that is not missed by a particular census can result in a substantial number of mistakes--enough mistakes to overwhelm the small number of people found by the survey who really were missed by the census. Based on simple examples and mathematical illustrations, Darga shows that even modest levels of error can mean that a majority of the people identified in the survey as missed by the census were not missed at all. This has extremely serious implications for the feasibility of the approach that has been proposed for measuring undercount, since the survey involves more than just small sources of error.

Evidence of Past Error

The book discusses three indications that the 1990 undercount measurements involved high levels of error.

First, the author shows that the individual survey measurements were inconsistent with the findings of the Census Bureau’s more reliable "demographic analysis" method. Even after the survey results were revised in 1991, the apparent undercount rates were 42 percent too low for black males, 25 percent too low for males of other races, 33 percent too high for black females, and 50 percent too high for females of other races. This provides strong confirmation for both fundamental shortcomings of the undercount survey: it fails to identify much of the undercount that is indicated by demographic analysis, while identifying many other people as missed when they apparently were not.

Darga also shows that several large undercount differences identified by the survey were totally spurious. Although most of the individual measurements of undercount cannot be definitively evaluated--we have very little reliable information about undercount at a fine enough level of detail--the stability of the sex ratio among young children makes it possible to definitively evaluate differences measured between young boys and girls. The author focuses on the eighteen segments of the population for which the 1990 survey indicated undercount differences of ten percentage points or more between boys and girls. (For example, for "blacks in noncentral cities of the Pacific states" the survey indicated that the number of boys under ten years of age should be increased by 31 percent, while the number of girls should be increased by only 6 percent--a difference of twenty-five percentage points.) The author finds that the census counts for these segments of the population do not deviate appreciably from the norm: the actual difference in undercount between boys and girls is essentially zero in each case tested. Only after applying the spurious undercount rates derived from the survey do dramatic deviations from the norm appear. It is dangerous to place much confidence in a coverage survey intended to tell us which segments of the population have higher undercounts than others if the survey indicates differences of ten percentage points or more where there is no real difference.

Finally, the Census Bureau's own unpublished evaluation reports on the 1990 undercount survey demonstrate high levels of error. Those reports documented serious problems with matching error, fabrication of interviews, ambiguity or misreporting of usual residence, unreliable interviews, unresolvable cases, and other problems. When the Census Bureau summarized these findings, it found that about 57 percent of the apparent net undercount identified in 1990 actually represented bias caused by such errors in measuring undercount. Subsequent analysis outside the Census Bureau suggests that the figure was at least 70 percent. Thus, many of the people who would have been added to the 1990 population count had not really been missed by the census. The adjustment process would have caused them to be double-counted, while many of the people who really were missed by the census were missed by the survey, too.

Conclusion

The proposed adjustment methodology cannot be relied on to improve the census, because it is based on a survey that is seriously flawed. Although virtually perfect accuracy is required to adequately measure a phenomenon as small and elusive as census undercount, a high level of accuracy cannot be attained. Significant segments of the population that are undercounted by the census are missed by the survey as well, while other people are mistakenly classified as missed by the census because of failures to match them with their census forms and other problems in implementing the survey. Those problems cannot be corrected by trying a little harder, or by increasing the sample size, or by making minor improvements in the methodology: they are the flaws inherent to any attempt to measure undercount by matching the individuals found by a coverage survey with the individuals found by the census.

Thus, instead of telling us which cities and which segments of the population have the greatest undercount, the proposed adjustments would primarily tell us where the greatest number of mistakes are made in the sample survey. And those mistakes--rather than accurate identification of people who were actually missed by the initial census enumeration--would form the basis for adjusting census totals and demographic composition for regions, cities, and communities. We expect a census to increase our knowledge about how our communities are changing and how they compare with other communities, but a census that incorporates faulty adjustments for undercount would increase our ignorance instead.