With This Test, Teachers Can Be Left Behind

A new report from the Montgomery County school system says the county's special-education teachers are having so much trouble administering an alternative version of the Maryland School Assessment to severely disabled students that nearly one-fifth of students tested last year failed solely because of errors committed by the teachers.

The finding is significant because of the difficulty Montgomery and other counties have had in meeting annual targets under the No Child Left Behind Act for special-education students. The population is one of eight statistical "subgroups" assessed under the law, which requires that each group meet the same academic proficiency goals as the school on the whole.

Special education was a factor for 15 of the 21 Montgomery schools that missed annual targets last year under the education law. Four elementary and four middle schools missed adequate progress goals solely because of special-education scores.

The report examines performance on the Alternate Maryland School Assessment, a test given to students with severe cognitive difficulties. The Alt-MSA is designed by teachers and tailored to individual children, in an attempt to assess them at their instructional level rather than at grade level. Alternative assessments were given to 686 Montgomery students last year.

Unlike the regular test, which is written by state officials and administered over the course of a few days, the Alt-MSA is a nearly year-long project.

Educators determine 10 objectives in reading and 10 in math that reflect adequate growth for a student. Teachers assemble portfolios of student work to illustrate student progress toward each objective. The process can take 60 to 80 hours of a teacher's time.

Proficiency, the standard by which schools are judged under the No Child Left Behind law, is attained if a child masters six or more of the 10 objectives.

Some school systems, including Prince George's and Baltimore counties, have demonstrated that proficiency is easily attained on the alternative assessment as long as the test is administered properly.

Only 9 percent of Prince George's students who took the Alt-MSA failed to attain proficiency in reading last year, compared with 36 percent of all students in the county, which means the alternative test had a positive net effect on the county's scores.

Montgomery, in contrast, has been vexed by the Alt-MSA. In 2006, only about half of students tested rated proficient. In 2007, proficiency improved to about 70 percent in each subject, compared with about 80 percent for all students.

On closer inspection, school system researchers found that many students were failing the alternative assessment because of mistakes in how the tests were written or submitted to the state.

Nearly one-fifth of students who took the reading and math tests never had a chance to rate proficient because half or more of the objectives assembled and submitted by their teachers were disqualified by state officials as "nonscorable," the report found.

An objective might be something as simple as identifying the title of a book when it is played from a recording, or recognizing a picture from a flashcard. Teachers were supposed to test students on each task as many times as necessary to demonstrate mastery.

Any objective could be rejected for one of six reasons. Some were not accepted because the material did not include the student's name or the correct date. Others were rejected because of missing or inadequate "artifacts," or samples of student work. In some instances, the work was judged not to have enough relevance to the statewide curriculum that is the basis for the test.

Such errors were largely responsible for the county's poor showing in 2006, when nearly half of all reading and math objectives submitted to the state were rejected. In 2007, the rate of rejection fell to less than 25 percent.

The report recommends several steps to improve performance. One is mandatory training for teachers that give the alternative test. Another is to focus technical help on schools with high failure rates. A third is to collaborate with school systems, including Prince George's, that are "successfully maximizing proficiency rates."