OSSE has winnowed from 128 to 35 the number of classrooms that it will ask an independent contractor to investigate for possible cheating on the 2011 DC CAS, the agency announced Thursday.
The 128 classrooms, spread across 54 public and public charter schools, represent less than three percent of classrooms citywide in which tests were administered last April. They were identified in a study last July by CTB/McGraw-Hill, publisher of the DC CAS, as having “inordinate numbers” of wrong-to-right erasures on answer sheets. The study, which wasn’t made public by OSSE until late December, said the data “may indicate inappropriate intervention on students’ answer documents by an educator.”
But in a statement, OSSE said only 35 classrooms met the expanded set of criteria the agency is now applying to determine whether a classroom’s test results should be fully investigated. These are: unusual test score gains by individual students from 2010 to 2011; wide variances or unusual patterns of scores within a classroom, and prior year’s test results that showed inordinate wrong-to-right erasures in that teacher’s classroom. OSSE said it consulted with an independent advisory committee of testing experts to develop the criteria.
“Erasures themselves are not an automatic flag,” said OSSE spokesman Marc Caposino.
The CTB/McGraw-Hill analysis of 2011 scores also cautioned against using elevated erasures as the sole criterion for investigation:
“We emphasize that the results from this study may be used in conjunction with other information to investigate whether inappropriate interventions may have taken place. Inordinate WTR [wrong-to-right] answer change rates, by themselves, may simply be coincidental and do not necessarily indicate inappropriate behavior,” the study said
OSSE has been taking bids for an independent contractor, or “test integrity vendor,” and will recommend a firm to city contracting officials early next week. The agency declined in December to name any of the 54 schools involved. Caposino declined again on Thursday to name the schools, saying the agency would not until the investigation was complete.
Last year, OSSE investigated just 18 of the 110 classrooms flagged for elevated erasures on the 2010 DC CAS. Based on the work of Caveon, a test security firm hired by DCPS, D.C. State Superintendent Hosanna Mahaley invalidated test scores last May in three classrooms (at Noyes, C.W. Harris and Leckie elementary schools). Two DCPS teachers were dismissed.
Caveon founder John Fremer said that Henderson never asked the company to use all the forensic tools at its disposal, some of which seem to be covered by OSSE’s expanded criteria. It is not known whether Caveon is bidding on the OSSE contract.
A USA Today investigation published last March found that classrooms in more than 100 D.C. public schools showed higher-than-average rates of erasures from wrong to right answers on the annual tests between 2006 and 2010 (It did not include public charter schools in its analysis). Henderson asked D.C. Inspector General Charles Willoughby to investigate the newspaper’s findings. His office is reportedly receiving assistance from the U.S. Department of Education.
Post columnist Robert McCartney excoriated District officials Thursday for the slow pace and limited scope of inquiries.