Those expressing qualms about the rigor with which Caveon Test Security investigated irregularities on the 2009 and 2010 DC CAS might want to look at the two $100,000 contracts it signed with DCPS. Caveon president John Fremer said there was no attempt to minimize problems or sugarcoat results. And he’s probably right. Caveon did exactly what DCPS asked it to do, which looks like it’s not going to be enough to get to the bottom of the erasure issue.
The first contract, signed in December 2009, includes a five-page outline from DCPS outlining the scope of the work, written by the school system’s Office of Data and Accountability, headed by Erin McGoldrick. Caveon’s mission was to review security protocols at each of the eight schools flagged for unusual rates of answer sheet erasures in 2009. It was also tasked to interview “a sample” of the teachers who administered tests that raised questions about wrong-to-right erasures. Then there’s this:
“The vendor will consult with D.C. Schools personnel as to which materials would be most useful for conducting the Test Security Investigations in a manner that minimizes demands on DC Schools staff,” McGoldrick’s outline said. “In some cases, it may be easier to interview an appropriate person rather than to have the staff synthesize materials if they are not maintained in a way that makes it easy to send them in advance of the interviews.”
The second contract, signed in February, to scrutinize schools flagged for security issues on the 2010 tests, does not include such specific directions. But a list of “deliverables” is largely the same as for the 2009 agreement.
The contract specifies a focus on 10 schools (not named) at a cost of $8,000 each. Whatever work Caveon was supposed to do on the 2010 CAS, it may have already done. It called for completion of site investigations by March 18. It’s reasonable to expect that the firm’s findings will be similar to what they found on the 2009 tests: no hard evidence of cheating.
My colleague Valerie Strauss argues here that only a more muscular investigation -- with subpoenas and sworn testimony -- will determine if these erasure “flags” are actually the result of cheating by staff or students, and whether they were urged by those above them to cross the line to compile improved scores.