The white paper "North Carolina Opportunity Scholarships: Countering a Flawed Duke Report," developed by Brian Jodice of Parents for Educational Freedom in North Carolina, is not only a victory for scholarship choice programs but for honesty in research — which, when it comes to school choice issues, is often negatively skewed in both its construction and execution.
Jodice's paper delineates the fundamental problems with the Duke report and other studies evaluating school scholarship initiatives like the recently-published negative assessment of the North Carolina Opportunity Scholarship Program. The Duke report and others like it are inherently flawed because they inaccurately depict students who participate in school choice programs as performing below expectations just because they have not skyrocketed to the top of the class during their first year in the program.
The new analysis sets the record straight on several fronts.
First, the methodology of the Duke report is poorly designed using both different comparison groups and different tests. For example, scholarship students who tend to be predominantly from lower-income families are matched against a national sampling of students who come from a cross-section of socio-economic backgrounds, creating an apples-to-oranges comparison. The study also integrates metrics such as "national average" without an explanation as to how the term is defined or calculated.
Jodice also points out that Duke's student test-performance data was only collected for one year, which is hardly enough time to make an adequate assessment of the program's results. Most studies show that it could take at least three years for aggregate test scores to show improvement. Furthermore, the report does not include a comparison of the students' test scores before and after their enrollment in the scholarship program, making it impossible to know if a student performed better or worse than he or she had previously.
That the Duke study completely ignores the good news from the North Carolina Opportunity Scholarship and other national school choice programs. For example, a recent study has shown that the likelihood of DC Opportunity Scholarship participants graduating from high school has jumped from 70 percent to 91 percent, a statistic that is a harbinger of positive outcomes for similar scholarship programs.
More than half of the 740 private schools in North Carolina participate in the program and more than 90 percent of first-year enrollees opt to enroll their children for a second year — all of which, arguably, is the most important statistic of all.
When parents buy into schools, their students are more likely to succeed.
There are three kinds of lies, the saying goes: lies, damned lies, and statistics. And while that adage may overstate the case, it can, at least, stand as a constant caution. First, to average people who read stories based on statistics, heed this generic advice: don't believe everything you read.
Second, to those who write about studies and statistics. Far too many news stories, opinion pieces, and issue analyses trumpet statistical "findings" that are overblown, misleading — sometimes wildly so — and that often draw conclusions that are unjustified, unfair, or flat-out wrong. Be less accepting of information that is handed to you, and more discerning in your assessment of it.
And third, the caution should be greatest to those who construct the surveys, create the data sets, and crunch the numbers. Your work may be "accurate" but does it carry a bias, is it fair, is it honest? Are there flaws? What have you missed? What have you omitted? Why?
By calling out Duke, Brian Jodice has done a public service for the people of North Carolina, and for all of us who are advocates for options, opportunities, and choice in education.
Jeanne Allen (@JeanneAllen) is a contributor to the Washington Examiner's Beltway Confidential blog. She is CEO and founder of the Center for Education Reform.
If you would like to write an op-ed for the Washington Examiner, please read our guidelines on submissions here.