What’s interesting about media fact-checkers is that, while they often prove to be subjective in their findings, they do allow others to objectively evaluate them since they append value judgments such as “true” or “false” to statements. I’ve previously noted two university studies, one at the University of Minnesota and another at George Mason University, that simply quantified PolitiFact’s results over a specified period and cross-referenced the results with partisanship,. The results were unsurprising to those who regularly marveled at PolitiFact’s reasoning—the Pulitzer Prize-winning fact checker rates Republicans as telling falsehoods much more often than Democrats (rates of 3:1 and 2:1, respectively). There’s really no other way to explain it other than the organization has, at a minimum, a serious selection bias problem.
Another issue I’ve often tackled is how the fact-checkers arrive at their ratings. They are often blindly partisan, grossly misinformed about complicated policy matters, or hypocritical, depending on which party’s ox is being gored. And sometimes there are just downright embarrassing errors. Getting at the substance of what fact-checkers say is a bit more complicated, but via Marginal Revolution, I see some Stanford political scientists had a go at simply trying to see how often fact checkers agree with each other. Here is the abstract from the paper:
That’s right—fact checkers don’t come close to producing ratings that would be acceptable by the standards of academic social science. And if you know what a garbage fire the issue of accuracy in social science is, that is really saying something.
I do love the understated conclusion that the media’s huge investment in disguising obvious partisan hackery under a facade of psuedoscientific labels about the truth suggest “difficulties in fact-checking elites’ statements may limit the ability of journalistic fact-checking to hold politicians accountable.”
You don’t say? It’s almost as we’ve spent the last two years watching the slow motion trainwreck of a hapless and incompetent media, having previously squandered much of its credibility for short-term partisan gain, repeatedly try and fail to rein in easily the two most dishonest and disliked presidential candidates in history, much to voters’ general indifference.
