JUST IN CASE you were worried that not enough supposedly neutral institutions and publications were gunning for George W. Bush, part of the international public health establishment just risked its reputation to join the fight. On Friday, the New York Times reported a new study in the British medical magazine Lancet claiming the U.S.-led invasion caused 100,000 Iraqi civilian deaths, a civilian mortality rate more than double that of the 14 months before troops went in. The subtext is clear: the overthrow of Saddam Hussein hasn’t made life safer even for Iraqis. The Lancet‘s editors were even clearer: they “fast-tracked” the article for publication because they thought it ought to affect judgments about “the governments . . . responsible for launching a pre-emptive war,” including, presumably, governments up for reelection five days after the study’s release. And the Times wasn’t exactly bashful either. They authoritatively state that the “study is scientific” and don’t quote a single expert skeptical of the finding.
Nor does the politics seem to stop at the article’s edge. There are at least three major reasons to be skeptical of the study’s findings. Here’s how the study worked. A team headed by a public health professor at Johns Hopkins asked a small sample of Iraqis about deaths in their households, and then extrapolated a national mortality rate from there. The first grounds for skepticism is a very unpersuasive methodology. Slate‘s Fred Kaplan points out that the study’s method only supports a 95 percent confidence in the conclusion that the war caused somewhere between 8,000 and 194,000 deaths, an extremely wide range. In other words, even taken at face value, the study says that the mortality rate may have gone up quite a bit but also may hardly have risen at all, or may have landed anywhere in between. But the study doesn’t explain this, its authors haven’t highlighted it in press statements, and the Times never mentions it at all. Now, conditions in Iraq probably permitted at most a very small sample, which compromises the validity of findings. But researchers know that going in, and this team chose to press ahead anyway with a study whose results would almost inevitably be so imprecise as to practically invite being overblown and overhyped.
Second, there’s a very good chance the study underestimates Iraq’s pre-invasion mortality rate. For one thing, to establish this rate the study focuses on a very unrepresentative period: the last year of Saddam’s rule, when his tastes for genocide and war were constrained by the threat of the very invasion these researchers now lament. Even so, the pre-invasion mortality rate reported in the study is curiously low given previous estimates. The study’s methodology may explain this too. For example, the Iraqis who were surveyed reported virtually no violent deaths in the last 14 months of Saddam’s rule. But as a rule, most Iraqis remain uncertain about the fate of loved ones who fell into the jaws of Saddam’s killing machine. Even more problematic is the estimated pre-invasion mortality rate among youngsters. The rate reported in this study is credible, the authors say, because it is “similar to estimates from neighbouring countries.” But that means it is three or four times lower than earlier estimates, so low that the researchers wondered if their respondents had under-reported pre-invasion deaths of infants and children.
They should wonder. Throughout the 1990s, a slew of public health studies insisted that international sanctions were causing infant and child mortality rates in Iraq dramatically higher than those in neighboring countries, resulting in hundreds of thousands of deaths. The new study does not repudiate those earlier findings and does not explain the stark disparity. This is especially curious because some of the most prominent of those earlier studies appeared in . . . the Lancet. This leaves the uncomfortable impression that when high infant mortality rates under Saddam made U.S. policy (sanctions) look bad, the Lancet published high rates; and when low rates under Saddam make U.S. policy (invasion) look bad, the Lancet publishes low rates.
There is also a good chance the study overestimates the mortality rate since the invasion. The Lancet‘s editors stress that a larger sample size was inadvisable given physical insecurity. But the study’s researchers were sure to survey in Falluja, far and away the most violent city in post-invasion Iraq. Falluja turned out to be such a wild statistical outlier that they offer two estimates, one with Falluja included and one with it kept out. But questions about just how representative the sample sites were go deeper than this. The researchers selected their survey sites households for such unclear reasons that we simply can’t extrapolate to the whole country with any confidence. What are the chances that they have over-sampled the most violent parts of the Sunni Triangle and under-sampled the calmest Kurdish and Shia areas? Without better statistics about population and violence, we can’t know, and neither can they. The fact that they don’t explain their strategy doesn’t build confidence in their research design or their conclusions.
In other words, we still know very little about how many Iraqis died, and how they died, under Saddam’s final years in power and how many Iraqis have died since his overthrow. For now, there is simply no well-established reason to believe that civilians have died at a much higher rate since the invasion than before it. And there is every reason to believe that the overthrow of a corrupt and genocidal regime will lead to many fewer deaths among Iraqis with every passing year.
But this period has claimed one victim for sure. A determination to defeat George Bush has led many newspapers, television networks, academics, NGO’s–and now even medical journals–to jettison their standards of fairness, restraint, objectivity, and integrity. Whether Bush wins or loses, it will take years for these institutions and organizations to regain the reputations they freely chose to throw away.
Gerard Alexander is associate professor of politics at the University of Virginia, and wrote A Lifesaving War in the March 29, 2004 issue of The Weekly Standard.
