Our Use of Conflict Death Tolls Compares Apples and Oranges

Simply put, estimates of conflict death tolls are really important. They help determine which conflicts gather media attention, where aid and diplomatic attention is focused, and shape historical memory. Therefore, it is a very significant problem that our use of death tolls is totally inconsistent.

Efforts to quantify deaths will always be extremely difficult in quickly evolving environments where there is a breakdown of record-keeping (if there ever was any), there is almost never a free press, and safety concerns prevent access to vast regions. Yet a completely avoidable problem is that “death toll” can mean very different things, and there is almost no effort to differentiate them.

Consider these two sentences from BBC reports:

June 23, 2016, “Nigeria Boko Haram: Scores of refugees starved to death – MSF”:

“The Islamist group’s seven-year rebellion has left 20,000 people dead and more than two million displaced.”

December 15, 2014, “South Sudan conflict: What chance of peace?”:

“At least 50,000 killed”

From these two reports, you would think South Sudan’s conflict was quite a bit deadlier than Nigeria’s, yet the reality is that we do not know. To some extent that is because there are large methodological questions about those numbers, but a bigger reason is that they are measuring different things. If you trace back those two figures, you discover that the South Sudan figure measures excess deaths, while the Boko Haram figure only measures violent deaths (The exact source of the 20,000 figure is somewhat unclear. Still, there is no evidence of any study ever measuring excess deaths in Nigeria; for example, no tweet has ever contained all of the words “Boko Haram excess deaths”).

Excess deaths refer to comparisons of mortality rates during the conflict with a baseline mortality rate, with the final figure supposed to represent the number of people that died that would not have had the conflict not occurred. This includes people that die from malnutrition because they are forced to flee their homes, illnesses that otherwise would have received medical care, or illnesses resulting from decreased access to clean water, none of which are picked up in violent death tolls.

The articles cited above make no effort to distinguish between violent and excess death tolls, but excess death figures are often multiple times larger than the violent death toll. Indeed, the majority of conflict deaths in South Sudan are not violent. These death toll figures get attached to conflicts as a representation of their scale, but by failing to distinguish between excess and violent deaths, the figures are extremely misleading. As bad as South Sudan is, Boko Haram’s insurgency easily could have led to more deaths than South Sudan’s conflict. While the lack of a study of excess deaths in Nigeria means we are unable to know, when we lump together excess and violent death tolls it then takes significant effort just to find out that we don’t know.

I spent some time going through the death toll figures most commonly referred to for various conflicts and attempting to trace them back to their sources. Many rely on questionable methodologies, but below I will clarify what the methodology is actually trying to measure.

Central African Republic, 5,000: This counts only violent deaths.

Darfur, 300,000: This counts excess deaths.

DR Congo, 5.4 million: Originating from a 2008 International Rescue Committee report, the figure counts excess deaths. Under 10% of deaths were violent.

Iraq: Likely due to U.S. involvement, the death toll of the Iraq War has been much more studied than many other conflicts, and there is no single most prominent figure. One of the most common is the Iraq Body Count’s 2013 figure of 112,017-122,438 violent civilian deaths. The Lancet found 654,965 excess deaths in 2006. However, it believed over 600,000 of those deaths to be violent.

Syria: Like Iraq, there are multiple figures used for Syria. The UN essentially stopped counting in 2013 with the toll at 191,369, only counting violent deaths. It has since offered estimates up to 400,000 but they are not official numbers. The Syrian Observatory on Human Rights has presented 271,138 documented violent deaths, with an estimate of 370,000 total violent deaths. Another prominent figure is the Syria Centre for Policy Research’s 2016 estimate of 470,000 excess deaths.

Yemen, 6,200: This figure comes from the UN and almost certainly only counts violent deaths.

There are a number of problems with conflict death tolls like these. Due to the extreme difficulty of obtaining information, they will always be highly unreliable. Numbers also tend to stick to conflicts even when they are years out of date, and there also is not enough emphasis on distinguishing between combatant and civilian deaths. Given these concerns, I think we should generally be more skeptical of the use of death tolls.

There also should be continued exploration of the merits of violent and excess death tolls. Excess death tolls may give a more complete reflection of the impact of conflicts. Violent death figures also tend to rely on documentations on violent deaths, leading to larger underestimates in contexts where information is particularly inaccessible. However, the methodological difficulties of calculating excess death tolls mean the numbers are arguably more unreliable. In some cases the difference between excess and violent death tolls will be relatively small, as is the case in Iraq and Syria. In others, particularly in poorer countries, it will be enormous. The failure to distinguish between excess and violent death tolls will be particularly influential in these cases. Cases like Nigeria are punished simply because no one has conducted an excess death study. That death toll would be likely be quite a bit higher, but right now it takes significant effort just to determine that the current figure of 20,000 counts only violent deaths.

There are extensive discussions of these issues in the scientific community, but outside of it there is almost no effort to even distinguish between excess and violent death toll figures. It took me quite a few hours just to figure out whether the numbers used in this blog were excess or violent death counts. To give another example, the Washington Post published an article in March specifically focused on comparing different estimates of Syria’s death toll but it fails to mention that some figures count violent deaths while others count excess deaths.

The failure to distinguish between excess and violent deaths is both avoidable and significant. If you only look at violent deaths, the DR Congo’s death toll is still unthinkably tragic, but it is under 540,000. That is significantly more than Syria but in the same ballpark, while the excess death figure rivals the number of Jews killed in the Holocaust. Of course, attention on conflicts is not directly proportionate to death tolls or even humanitarian concerns more generally, but death tolls still do play an influential role. To the extent that it does play a role, we should actually know what the numbers we are using mean.