Recent news coverage of a report by Toronto Public Health (Air Pollution Burden of Illness in Toronto: 2004 Summary) would suggest the answer to this question is cut-and-dried. The Globe and Mail certainly didn’t imply much doubt with its headline: “Death Toll from Dirty Air Rising, Study Finds.” Other newspapers, such as the Vancouver Sun and the Vancouver Province, ran variants of a Canadian Press article that focused on an absolute value, rather than a trend, but the depiction of certainty was the same: “Toronto Air Pollution Behind More Deaths than Originally Estimated: Study.”

But where do such “Burden of Illness” estimates come from? Are there 1,700 autopsies declaring air pollution as the cause of death? Are 6,000 people admitted to hospital each year with a diagnosis of “air pollution” as the cause? Even the author of the Toronto Public Health report admits there are not, writing: “Given the nature of these studies, it is not possible to point to a specific person who has been hospitalized due to a breathing or heart problem and determine that air pollution was the reason that this person was admitted to hospital.” Still, the author of the report, and those writing the headlines of the news coverage, clearly felt that the absence of actual sick or dying people was no obstacle to declarations of grave danger and wideranging public policy prescriptions.

What was not given much attention is how the body-count estimates are derived and whether the methods produce valid and plausible results. The estimates are based on small statistical correlations observed between air pollution and the number of people that die on a given day. But the devil is in the mathematical details of how these studies are performed, and the media have been notably unskeptical in taking the studies at face value. As is shown below, even the results of the background studies used to relate air pollution exposures to health risk don’t support the conclusions that regulators and their media allies have drawn from them. But you have to read past the studies’ executive summaries to find that out.

First, the air pollution risk factors for exposure to fine particulates used in the Toronto Public Health study are not produced by direct measurement, but rather, through a type of epidemiological study that follows the health status of a group of people over time and then looks for correlations with exposure to air pollutants. In this case, the risk factors used for the Toronto population were, “. . . risk coefficients for chronic exposure to particulates obtained from the re-analysis by the Health Effects Institute of the Harvard Six Cities Study and the American Cancer Institute Study of Particulate Air Pollution and Mortality.” Using risk factors from these two studies, the Toronto Public Health report estimates that in 1999, over 1,200 people (out of the claimed total of 1,754 deaths from all pollutant exposures) died from exposure to fine particulate pollution (particles under 2.5 microns in diameter). What does a closer look at these two studies tell us?

The American Cancer Society (ACS) Study

Some of the results of the American Cancer Society study that was adopted by Toronto Public Health suggest that the apparent association of particulate pollution with mortality is a statistical artifact, rather than a real cause-effect relationship.

For example:

• There was no association between exposure to fine particles and death for persons with more than a high school education, for women, and for people between the ages of 60 and 69.
• Fine particle exposure was associated with increased mortality for former, but not current smokers, or those who never smoked.
• Fine particle exposure was associated with increased mortality for people who said they were moderately active, but not for people who said they were either sedentary or very active.

These odd variations in the statistical relationship between fine particles and mortality appear to be biologically implausible and suggest that other factors besides pollution would better explain the results.

The ACS study produced other odd results:

• Fine particle exposure was not associated with an increase in lung cancer mortality in the Health Effects Institute re-analysis, but was associated with an increase in mortality due to other cancers. Higher levels of fine particles also were not associated with an increased risk of mortality due to respiratory disease. These findings are surprising, given that particulate pollution would be expected to exert its effects through the respiratory system.
• When population change was added into the statistical model, the apparent effect of exposure to fine particles declined by two thirds and became statistically insignificant. The hypothesis here is that cities that were in economic decline, in this case Midwest “rust-belt” cities, were more likely to lose population. These cities also tended to have higher air pollution levels. The people who leave to find work elsewhere tend to be the healthiest. Thus, what appeared to be an effect of particulates was more likely a result of differential migration.

Another concern with the ACS study is that information about participants’ health-related behaviors and status, such as diet, body-mass index (BMI), and smoking were assessed only in 1982 when they entered the study, but not afterward. If any of these factors changed after 1982, and if the changes were correlated with pollution levels, then the study results would suffer from additional uncontrolled biases. For example, the ACS study followed participants from 1982 to 1998. If people living in areas with higher pollution were also either more likely to gain weight, or less likely to stop smoking when compared with people in lower-pollution areas, researchers could mistake an effect of body weight or smoking for an effect of air pollution. The rate of obesity increases or smoking decreases and the likelihood of living in an area of greater air pollution are probably positively correlated through their common association with socio-economic factors such as income and education, suggesting this is a concern worth additional investigation.

Long-term studies are based on the idea that long-term exposure to elevated pollution causes the development of cardiovascular disease or cancer. These diseases generally develop over a period of 15 to 20 years between exposure and manifestation of disease, suggesting that pollution exposure should be measured during a time period years before the health effect appears. Yet the ACS study of pollution measurements occurred around the same time the study began in the early 1980s, when the range of particulate pollution levels was about four times lower than during the 1960s. If it was these earlier high particulate pollution levels that actually caused the health effects, then the real effect of air pollution would be one-fourth that estimated in the ACS study. This is because studies like the ACS study estimate the concentration-response function for the health effects from particulate pollution based on the range of particle levels across cities in the study. If this range is actually four times greater than the range used in the American Cancer Society study, then the health effects of a given increase in PM would be one-fourth of what the ACS study estimated.

The Harvard Six Cities (HSC) Study

The HSC study compared chronic mortality data with annual-average fine particle levels in six cities located in the Midwest and northeast of the United States. Fine particle measurements were collected from the late 1970s through the mid-1980s, and mortality data were based on a 14- to 16-year follow-up of about 8,000 participants.

HSC reported, after adjusting for other health-related factors such as smoking and educational attainment, that there was a 26 percent increase in risk of death between the city with the highest mean PM2.5 level (29.6 µg/m3) and the lowest (11 µg/m3). This works out to a mortality increase of 14 percent for each 10 µg/m3 increase in PM2.5--substantially larger than that found in any of the other long-term mortality studies.

But there is also evidence that the HSC results suffer from the failure to account for non-air-pollution causation. For example, HSC did not account for the physical activity levels of the study participants, yet exercise is strongly correlated with health. It turns out that levels of physical activity in the six cities are inversely correlated with particulate levels in these cities. HSC might therefore have attributed to air pollution a health effect that was actually caused by lower physical activity levels. Like the ACS study, there was no association between fine particulate pollution exposure and mortality in people with more than a high-school education. HSC also found that greater exposure to fine particles was associated with a statistically insignificant decrease in mortality due to respiratory causes specifically.

The HSC study was based on fine particle levels measured concurrent with the beginning of the follow-up period, even though mortality was due to diseases with long latency times. Like the ACS study, the HSC study might therefore have inflated the apparent effect of fine particles on mortality, compared to an assessment based on much greater fine particle levels in the two decades leading up to the HSC follow-up period.

The Toronto Public Health study seems to have ignored another study of particulate matter and health that followed a group of 50,000 veterans with high blood pressure for 21 years and correlated their health status with pollution levels where they lived. Even though pre-existing high blood pressure should have made this group more susceptible to any health effects from air pollution, the study found no relationship between air pollution and mortality.

Canadians are very concerned about air pollution. Evidence of lethally high air pollution concentrations from decades ago, and oft-cited comparisons of modern air pollution exposures to cigarette smoking have put air pollution control high on most peoples’ radar. This would be perfectly sensible if air pollution ranked highly among the myriad of other risks we face, such as death from cancer, heart disease, contagious disease, accidents, and so on, but the evidence suggests that air pollution risk is far lower than any of these major risk factors.

While it may be instinctive to associate polluted air with major health problems, our instincts can mislead us if we devote scarce public health resources toward risks that are small, of low probability, and costly to reduce, while other risks that are larger, of higher probability, and less expensive to reduce get short shrift. And we are particularly easy to mislead when news coverage treats new studies of risk superficially--devoid of context and devilish details.

References

Canadian Press (2004). “Toronto Air Pollution Behind More Deaths than Originally Estimated: Study” (July 8).

Mittlestaedt, Martin (2004). “Death Toll from Dirty Air Rising, Study Finds.” The Globe and Mail (July 9).

Pengelly, L.D. and J. Sommerfreund (2004). “Air Pollution-Related Burden of Illness in Toronto: 2004 Update, Technical Report.” Toronto: Toronto Public Health. Available digitally at http://www.city.toronto.on.ca/health/hphe/pdf/
air_and_health_burden_technical.pdf.

Schwartz, Joel (2003). “Particulate Air Pollution, Weighing the Risks.” Washington, DC: Competitive Enterprise Institute. Available digitally at http://www.cei.org/pdf/3452.pdf.

Yaffe, Barbara (2004). “Air Pollution Burden of Illness in Toronto, 2004 Summary.” Toronto Public Health (July). Available digitally at http://www.city.toronto.on.ca/health/hphe/pdf/
air_and_health_burden_illness.pdf.

Kenneth Green directs the Centre for Studies in Risk, Regulation, and Environment at the Fraser Institute. Joel Schwartz is a visiting fellow at AEI.