Tuesday, October 24, 2006

An important question about the Iraqi mortality study

I was thinking of doing some more explanations of statistics today... of explaining how and why random sampling works, and how and why it gets you accurate numbers.

You know what? Screw that.

Because there's something more important.

Dr. Burnham's study, published in Lancet, suggests that over 600,000 Iraqis have died that would have lived had the United States not invaded Iraq.

The following are not matters of opinion; they are solid facts.

  1. The pre-invasion death rate matched the expected statistics from other agences for the same time frame, so there is no reason to discount the death rate as too low

  2. Clustering is a useful and well established method for gathering data in places like Iraq, and the selection of the clusters was done in a way that should give a random sampling of the populations

  3. 47 clusters is sufficient to get the level of precision the study authors wanted

  4. The calculations and inferences made are proper

  5. While the confidence interval around the number of deaths is relatively wide, it's properly calcluated

  6. It woould be extraordinarily improbable for the researchers to have been able to sample the number of people they did, and see the number of deaths that they did, unless hundreds of thousands of people have died in Iraq

Now, I know how statistics work, but I'm not an expert in experimental design. I can vouch for some of these points, but some are areas outside my expertise. So, how can I state that they are all solidly established facts? The answer is simple: peer review. Before the article was published in Lancet, four reviewers looked over the methodology. They dug in to see if there were any flaws in the experiment, or how it was run. They all suggested that the article be published with some minor changes (none of which affected the outcomes).

Anyone who claims that the pre-invasion death rate was obviously too small, the study was too small, or that the samples were poorly taken, or that you can't extrapolate hundreds of thousands of deaths from the several hundred that the researchers found, is simply wrong. Not "expressing a different opinion"; people who say these things are wrong. They are making criticisms that are contradicted by facts.

Now, does this mean that the study has perfectly accurate results? No. It could still be wrong, for reasons that are hidden to us. The sampling was done in a random manner, but I heard that one set of researchers said that it could have put people closer to the 'action', since it was based upon streets intersecting with main streets. Is this valid? We really don't know. The only way to know is to get a more accurate count of the deaths. There are also people who had different estimates for the pre-invasion death rates. Could they have had a better estimate? It's possible.

Which brings me to my main point.

There have been many, many people who have insisted that they have, or someone else has, debunked the study. They've all declared that the results can't be trusted, that the real numbers have not been found.

Think about that for a moment, and try to understand what an astonishing thing it is to say.

"You can't say that there have been 650,000 excess deaths! The fact of the matter is, we don't know how many excess deaths there are!"

Is there something missing from that statement?

I suppose it depends on whether you think that the number of Iraqis who have died as a result of this invasion is important. I mean, if you just don't give a damn about a bunch of dead Iraqis, then I suppose there's nothing missing. All you have to do is insist that a report that might make you look bad is wrong.

But if you care, if you think the number of deaths matters, if you truly value human life, if the thought of hundreds of thousands of deaths sickens you, then there's a huge hole in that statement, one that you probably think is obvious.

Even if you think Dr. Burnham's study is wrong, even if you think it's been "debunked", it shows that it is absolutely essential for us to find a better estimate.

Because even if the study is flawed - and I don't believe it is, but I'm willing to pretend, just to make a point - it shows that if you grab the wrong group of people, it looks like Iraq has lost hundreds of thousands of people. It looks like the cost of this war has been much, much too high. Wouldn't anyone who cares about life insist on trying to find the real numbers? So why haven't they?

Why has there been so much talk about proving this study wrong, and so little effort to find other, better numbers, and prove them right?

Have I made this point before? I don't remember, but it's worth repeating. Someday, people might look back on this, and see it as a warning sign. They might say that we had two warnings that the war was spiraling out of control, that things were much worse than we could imagine. And maybe they'll look at us, and ask "what did you do when you received those warnings?

I can answer truthfully: I paid them heed, and I tried to convice others to do so. I emphasized that they might be false warnings, but I said, over and over, that we needed to find the truth.

Can those who have denounced this study say the same?

Comments: Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?

Weblog Commenting and Trackback by HaloScan.com