I want to think well of the D.C. Office of the State Superintendent of Education (OSSE) and the superintendent herself, Hosanna Mahaley. OSSE has taken heroic measures in the past, under Mahaley’s predecessors, to make sure parents, voters and taxpayers know what is happening inside D.C. schools. It has released detailed annual test results and, in response to Freedom of Information requests, given journalists evidence of tampering found in test answer sheets in more than 100 schools.
That isn’t happening any more, as seems evident from an OSSE press release sent out Dec. 23.
Along with several other journalists, I filed a Freedom of Information request last spring asking for the ratio of wrong-to-right erasures on the D.C. Comprehensive Assessment System tests taken then. This is the key statistic that in previous years revealed that tests were showing far more instances of wrong answers being erased and replaced with right answers than usually occurred in those grades. Experts were nearly unanimous in saying this indicated adults had persuaded students to change answers, or had changed them after the students went home.
I checked occasionally on the progress of my request. In September, OSSE spokesman Marc Caposino indicated that the data was ready and I would get it after Mahaley returned from a trip to Brazil. Months passed and nothing happened. Often, there was no response to my requests for a update. Other reporters had the same experience. We had similar difficulty getting any information on the progress of an investigation of the erasures by the D.C. inspector general.
Now, instead of releasing its erasure data for 2011 as promised, OSSE has done something that will delay the release of that data for several months, if not years. Is this an intentional effort to deny parents, voters and taxpayers the information they need to judge the integrity of D.C. test results? I don’t know. What do you think? Is it significant that OSSE released the news just before a major holiday, which some organizations have done to deflect attention from information that does not make them look good?
The Dec. 23 press release, entitled “OSSE announces enhanced test security measures for D.C. public schools and charters,” said that “to ensure the validity” of the 2011 DC-CAS results, it “today issued a Request for Proposal soliciting vendors to investigate and assess individual classrooms.”
OSSE is going to hire another consultant to do another investigation. That will likely keep parents, teachers and students in the dark for a long time.
Caposino told me that OSSE did not intend to release the information on the eve of a major holiday. He said approval from the City Contracting office took weeks and they made it public as soon as that happened. “These measures are not being taken to hide anything, instead they are to make sure everything we report has been thoroughly reviewed and verified before anything is made public,” he said.
The press release said OSSE recently “created an enhanced set of statistical measures to more thoroughly analyze and assess student-level gains in achievement from 2010 to 2011. These statistical measures include student erasures from an incorrect to correct answer; student erasures for consecutive or multiple years; and in-classroom variances.”
So, if they have these new measures and the 2011 erasure data, which they say they do, why not let parents, educators and taxpayers concerned about the validity of the D.C. testing system see them?
William Schafer, former director of student assessment for the Maryland state education department and a retired University of Maryland education professor, is one of the four members of the OSSE technical advisory committee who made suggestions on how to deal with the test security problem. He said the committee was not shown the 2011 erasure results but got the impression that OSSE officials thought they revealed a problem. Caposino said the committee got an overview of the erasure results and a spreadsheet of third-grade classrooms that were flagged.
Schafer said the committee did not do its own report but told Tamara Reavis, OSSE director of assessment and accountability, that her office should decide what statistical methods can be used to accurately calculate signs of test tampering. He said they also advised that everyone in the school system working with tests should be made to understand precisely what they may and may not do to ensure security. The committee told OSSE to do whatever is necessary to encourage school employees to report any activities that seemed to compromise test security.
Schafer said he had not seen the request for proposals that OSSE just issued. To me this sounds like the District’s previous hiring of the Caveon testing security company to look at previous erasures that suggested that children in some schools changed wrong to right answers four or more times as often as the average for the rest of the District.
Caveon visited some schools and did some interviews. In nearly every case it accepted as “plausible” the explanations it got from school personnel for the wrong-to-right erasures – for example, that students were trained to check their answers. That explanation makes no sense to me or to several veteran educators I have asked about it. Do third-graders really check their answers and suddenly realize that many are wrong? Or are they just eager to get the test over and go to recess?
Caveon did not interview students or compare students’ test scores with their scores on previous tests, which might also have indicated tampering by teachers or principals bent on showing score improvements. A much more thorough Georgia state investigation of test tampering in Atlanta uncovered massive fraud by principals and teachers.
So why not release the D.C. 2011 erasure results now? Is it possible that OSSE officials do not want a repeat of what happened the last time it released erasure results from previous years to reporters for USA Today and to my colleague Bill Turque. The USA Today report last March, which included precise data for more than 100 schools, showed incredible numbers of changed answers. That created widespread doubt about the D.C. results, forced D.C. Schools Chancellor Kaya Henderson to ask the D.C. inspector general to investigate, and created great interest in what the 2011 erasure results would show.
I am sure the results of the new investigation will be interesting when we finally get to see them. But I strongly suspect that anyone able to read this newspaper would be able to provide a intelligent and useful analysis right now of what is going on, if only OSSE would release the data that up to now it has not shown to even its own technical advisers.