DCPS released its latest round of reports late last week from Caveon Test Security looking at possible cheating on the DC CAS. But they are not likely to quell voices--including Washington Teachers’ Union president Nathan Saunders, and more than 4,000 signatories of a petition sponsored by Teachers & Parents for Real Education Reform--calling for a more thorough investigation.
One of their major concerns is that Acting Chancellor Kaya Henderson didn’t ask Caveon to use all of the forensic tools at its disposal. Skeptics also note that she did not ask Caveon to look at all of the more than 110 classrooms across 41 schools flagged by CTB/McGraw-Hill for elevated levels of wrong-to-right erasures on the 2010 test. DCPS asked the firm to look at just ten schools.
The one case of cheating confirmed by Caveon was at Noyes Education Campus, where a teacher admitted to pointing out incorrect answers to students who appeared to be struggling. This is at a Pre-K-8 school with 16 flagged classrooms and an erasure rate more than six times the District-wide average in 2010, according to data obtained by USA Today.
Caveon’s report on Noyes, released in heavily redacted form by DCPS along with accounts of its inquiries at nine other schools (Leckie, Harris, Beers, Davis, LaSalle-Backus, Plummer, Truesdell, West and Whittier) shows four other individuals interviewed: the principal, former principal, test coordinator and a teacher who served as a proctor. In the only interview summary not completely redacted, the proctor said, “Everything was done by the book” in administering the 2010 DC CAS.
Caveon’s conclusion: “For the most part, testing procedures at Noyes seem good, and nothing out of the ordinary was noted from the interviews and reviews of test documents.”
Caveon founder John Fremer said he was doing exactly what his client, DCPS, asked. Had it asked for more, he said, more could have been done. The Utah-based firm could have analyzed answer sheets at greater depth--far beyond erasure rates, which Fremer says are the crudest and least reliable marker for possible testing misconduct.
It could have searched for patterns of collusion, looking for unusual levels of agreement on answers among students seated near eachother. It could have checked for logical inconsistencies in answer patterns, determining if students were doing unusually well on the harder questions while getting easier ones wrong. It could have gone back and looked at prior-year performance by students, or the performance of classrooms under certain teachers in the past.
“I could do everything you could ever want done,” Fremer said in an interview last week.
That would have required giving Caveon access to raw test data from CTB/McGraw Hill, publisher of the DC CAS, which scanned all the answer sheets looking for erasure patterns. But since CTB/McGraw-Hill was hired by OSSE, it would be up to the state agency to acquire it.
Henderson said OSSE has never voiced the need for deeper scrutiny.
“If anyone has evidence, I’ll act on it,” she said. “But I can’t conduct a witch hunt based on speculation.”
Even then, looking at classroom behavior only goes only so far. It doesn’t completely cover what happens in a school before or after a test, when questions can be leaked ahead of time or answer sheets can be tampered when they are handed in. That requires people reporting what they’ve seen. And critics of the DCPS investigation said that requires a whole different level of scrutiny.
After USA Today published its investigation in late March, Henderson asked D.C. Inspector General Charles Willoughby to review the newspaper’s findings. But some critics say the IG, who is appointed by the mayor, is not positioned to conduct a proper investigation. Earlier this month, Washington Teachers’ Union president Nathan Saunders wrote to Henderson that it would “greatly benefit the perceived integrity of this investigation if it were handled by a government agency outside of the District.” He proposed that the U.S. Education Department’s Inspector General be asked to probe the matter.
Saunders’ sentiment has been echoed by more than 4,000 teachers, parents and other stakeholders who have signed the petition asking that the Education Department and/or the U.S. Government Accountability Office (GAO) conduct an inquiry that would include “sworn, public testimony from principals, administrators and the former and current District and state leadership, as well as forensic analysis of student scores compared to other measures of what students know.”
“It must address all specific allegations of erasure and falsification of answer sheets, as well as any district actions that might have encouraged cheating, or that were taken to cover it up,” the petition said.