The MCPS claim about the shrinking grade 3reading gap.
MCPS slide/Singam
Montgomery County Public Schools (MCPS) spokesperson Dana Tofig has been quoted in the media as saying MSAs “… have proven to be pretty unreliable measures.” This reality hasn’t precluded MCPS and the state from claiming, in independent presentations, that the achievement gap is narrowing.
Leslie Wilson, Assistant State Superintendent, Division of Accountability and Assessment, in a presentation dated July 21, 2009, asserts the gap has narrowed in Maryland. In a presentation dated a few days earlier, MCPS asserts that the “MSA Grade 3 Reading Gap Shrinks 22 Percentage Points,” and the “MSA Grade 3 Math Gap Shrinks 16 Percentage Points.” In about seven years, the achievement gap, as measured by the MSAs, has shrunk by double digits.
Cause for celebration? Not so fast. After all, MCPS did assert the MSAs are “… pretty unreliable measures.”
There are reasonable indications that the MSAs are indeed “… pretty unreliable measures,” not the least of which is a recent column by this author. If indeed there is a substantial narrowing of the achievement gap, it is reasonable to expect other tests to reflect this laudable achievement.
For example, MCPS administers the TerraNova (TN/2) test to second graders. To be precise, “since 2000, MCPS has administered the CTBS to students in Grade 2 on a yearly basis to provide comprehensive measurement of basic skill achievement in five areas: reading, language, mathematics, language mechanics, and mathematics computation. During the 2005–2006 school year, MCPS [did] replace the TerraNova Comprehensive Tests of Basic Skills (CTBS) in Grade 2 with the TerraNova Second Edition (TN/2).” As the system further acknowledged in 2006, “both the CTBS and TN/2 are nationally norm referenced tests that compare performance of MCPS Grade 2 students with all Grade 2 students in the nation (the norm group). However, the TN/2 uses a more recent and inclusive student sample to establish norm group performance (CTB/McGraw-Hill, 2002).” The system interprets TN/2 performance in terms of normal curve equivalent (NCE) scores and reports scores in terms of percentages of students scoring above the national average (50th NCE).
In 2006, 47% of African-American students scored at the 50th NCE or higher in reading. By 2009, it had risen to 52%. Whites, in comparison, went from 77.8% to 82.2%, in the same period. The rise in scores being of similar magnitude, one can assert that the claims of the achievement gap in reading being substantially reduced are probably greatly exaggerated.
There a number of lessons to be learned through this exercise. Not the least among them is the need to look at a rigorous analysis of more than one indicator before concluding anything.
The second conclusion is that the MSAs are, as MCPS asserted, “… have proven to be pretty unreliable measures.” It is time the Maryland State Department of Education took a harder look at the MSA test and determined if it effectively serves the intended purpose.
