In case you haven’t noticed, there’s a lot of junk floating out there in the gun debate. I’m not just talking about name-calling, poisoning the well, or other nasty rhetorical business practiced by both sides. I’m talking about the statistical garbage that gets in the way of any halfway decent debate on gun violence in America.
Tidy charts crisscrossing leading social media outfits hide misleading information quite well, and sadly substitute for statistical analysis in fact-free comment exchanges and Twitter hot takes.
Take, for instance, this doozy created by an organization called Safe Home, and shared hundreds of thousands of times on Facebook and Twitter. At first glance, there appears to be a strong negative correlation between gun deaths in a state and the stringency of gun laws. But drop in on the methodology section of their website, and the sloppy empirical work rears its ugly head. The “stringency score” presented in the graph turns out to include as a factor the “gun death rate” itself. (See here.). Hence the astonishing finding that gun deaths are highly correlated to ... gun deaths.
That’s just the tip of the iceberg. The portion of the stringency score that deals with gun control laws is just arbitrarily weighted, with no explanation as to why some laws are more important than others. For example, “Carry Laws That Give Law Enforcement Full Discretion” is rated five times more heavily than “Secure Firearms from Domestic Abusers,” which is both counterintuitive and unexplained.
A more honest approach, albeit still crude, would be to compare each individual policy to state-level firearm death rates, controlling for a variety of other factors. A research team from Boston University, Columbia University, and the University of Bern undertook this sort of analysis in 2016, and found that few popular measures had any meaningful correlation with state gun murder rates.
A plethora of popular proposals, ranging from mandatory firearm locks to child lock restrictions to “assault” weapon and large magazine bans had no measurable impact at all. Background checks for guns and ammunition had large measured impacts, but these findings are hard to read into because, at the time of the study, only two states had universal background checks for all guns while only three had ammunition checks.
That’s not to say that gun control advocates are the only perpetrators of statistics abuse. Gun rights proponents often point to sky-high murder rates in inner cities as evidence that gun control in said jurisdictions don’t work. This ignores, of course, the ease of transporting weapons across state and jurisdictional lines, and fails to control for other factors like population density, poverty, and gang activity. Sure, Chicago’s murder rate is high, and it has strict gun control, but would it be even higher if there was no gun control enforced in the city and the state of Illinois? Both sides can try all they’d like to divine the answer, but a study with no controls is an exercise in futility.
Part of the problem is the presumption of “common sense” in the debate. To many, it axiomatically follows that since semi-automatic weapons can kill people at a far faster rate than non-semi-automatics, banning them would cause there to be fewer deaths. Even if killers substituted for non-banned guns, mass shootings would suddenly become less deadly. This is so “common-sense” that few have bothered to check the data on this claim.
According to data gathered by Mother Jones, the average mass shooting or spree in 2017 not involving a semi-automatic weapon was deadlier than ones using at least one semi-automatic weapon. And, in four of the past six years, the total death count for non-semi-automatic mass killings exceeded the semi-automatic count.
A lot of substitution can and does happen between guns. Proposed semi-automatic bans often make substitution even easier by using cosmetic features on firearms as a basis for legality.
Regardless of any future studies that get published or retracted, the gun debate will inevitably rage on. A better data policing effort on both sides, however, will make for a more sober debate with more promising outcomes.
Calling out your own side for creating pretty charts that hide ugly methodologies will leave everyone better informed. And when all is said and done, you can rest assured that your $3,000 statistics course in college was well-worth the price.
The gun debate doesn’t have to be a junkyard of flawed assumptions and pseudoscience, if we all put on our janitor’s hats and mop up the mess.
Ross Marchand is an economics writer based out of Washington.
If you would like to write an op-ed for the Washington Examiner, please read our guidelines on submissions here.