One of the difficulties facing citizens interested in evaluating the performance of their public officials at all levels of government is the multiplicity of often-conflicting claims made by advocates on all sides of most issues. Such claims are frequently based on data-driven studies and sound so persuasive.
There is a simple solution here, and all it requires is making a simple request: Let us see your datasets, please. “Datasets” include the raw data used in a study, as well as the tools, techniques and assumptions employed in the analysis. Advocates using credible, data-driven studies will eagerly provide their datasets because they know the credibility of their claims rests upon the methodological transparency of their study. Such advocates know they enhance their credibility by inviting others to examine their work. On the other hand, advocates with something to hide won’t for fear the games they play with the data will be exposed.
The importance of this fact was driven home for The Examiner in an exchange earlier this year with Brian Edwards, spokesman for the Montgomery County Public Schools. After reading a MCPS Web site claim about the mean SAT average scores of students in that system, The Examiner asked for the median and modal averages. Recognizing that individual student names could not be released, we asked for the data with numbers assigned in place of student names, an accepted technique among statistical analysts for protecting confidentiality.
Edwards denied The Examiner’s request and claimed MCPS analysts have no interest in median and modal averages. He further claimed that the SAT data was owned by the College Testing Board and could not be released by local school systems. Putting aside the clear preposterousness of such a response, The Examiner has resolved that whenever a public official makes a claim about his or her performance or promises or those of a public agency that is based upon a data-driven analysis, this newspaper will request the datasets behind those claims and will make them public via The Washington Examiner Community Action Network Web site.
The Washington region has one of the most highly educated populations in the world, so The Examiner’s readers include in their ranks thousands of skilled analysts, auditors, accountants and experts in countless issue areas. Together, The Examiner and its readers can bring a new level of accountability and transparency to the discussion of public policy issues at all levels of government. Together, we can sort out who can and cannot be believed in government.
In other words, Mr. and Mrs. Public Official, you’ve been warned.
