At the Beacon School in Manhattan, the teachers and administrators thought they had resolved, at least to their satisfaction, the long national debate over how best to assess students' work. Instead of the usual multiple choice tests and letter grades, they decided when they opened the school in 1993 to treat their diverse student body -- 26 percent from low-income families -- like graduate students. The high schoolers would do long projects and defend their work before faculty panels.
Modern educators call this assessment by portfolio, and Beacon, a public alternative school, became a national model for its advocates. Portfolios are collections of student work, the term derived from the carrying case of paintings or drawings that an artist presents as proof of her talent.
Many of the best teachers I know assess students this way, at least part of the time. They put great emphasis on special projects that call on many of the skills and concepts being learned in the course. This has produced in some high schools, even in low-income neighborhoods, a graduate school atmosphere in the senior year, with students doing work at least as good as many of the seminar papers I listened to when I was struggling to get a master's degree in East Asian Regional Studies a long time ago.
But portfolios as an alternative to the kind of standardized testing that bothers so many educators have not been doing well lately, and I think it is important to figure out why. Many people -- including me -- would like to create a world where standardized tests and portfolios could co-exist, the tests serving as a quick check on what has been learned and the portfolios giving students a sense of how their lessons fit together. At the moment there is a tension between the two approaches that has not been successfully resolved.
When portfolios were at their peak at Beacon, seniors would present portfolios of essays, lab reports, problem solutions and research projects of the past three years -- specifically, three science projects, three history projects, four English projects and three foreign language projects. But when New York state revived the regents tests as the prime determination of who would graduate, Beacon was forced to reduce the number of projects and cut the time for assessing them. Principal Stephen Stoll told me the biology course that, for instance, a year ago had 70 labs now has only 30, because students need more time to learn terms and concepts that will be on the regents test.
As the movement to raise high school graduation standards has grown in the 1990s, and the No Child Left Behind law has required states to use standardized tests to rate their schools, the idea of basing promotion and graduation decisions on portfolios has fallen out of fashion as swiftly as slide rules gave way to calculators. Some schools have tried to keep portfolios as a tool for classroom teachers, but even the most ardent advocates have acknowledged that samples of student work cannot easily compete with standardized tests in quickly and cheaply determining the overall performance of a school or a school district.
"If the goal is simply to sort, stratify and rank, portfolios add little if you already have test data," said Monty Neill, executive director of the FairTest, the Cambridge, Mass.-based organization that opposes standardized testing. "If the goal is rich feedback at individual or school level, portfolios of some sort are indispensable while tests are of minimal use as they provide far too little information."
The notion of authentic assessment -- judging a child's work first hand rather than summing it up with a letter or a number -- goes back to the beginnings of the progressive education movement a century ago. Even then portfolios were considered time-consuming, but many teachers and students liked them and they became a key part of the alternative public schools that were born in the 1960s and 1970s. At Central Park East Secondary School in Manhattan, Deborah Meier and other progressive educators began to judge low-income, inner city students based on collections of their best work and oral examinations, and found that if they did well on those alternative assessments, they got into college and did well there.
The National Writing Project, begun in 1974 at the University of California at Berkeley, stemmed from a similar notion that the process of writing, with repeated drafts and frequent editing, was more important in teaching than applying a grade, and those series of drafts would be all an assessor would need to judge the student.
The portfolio idea gained strength in the 1980s. Drew Gitomer, vice president for research at the Educational Testing Service near Princeton, N.J., worked on the Arts Propel project with Howard Gardner and Dennie Palmer Wolf of Harvard's Project Zero. "We explored the idea of portfolios in writing, music and art -- the latter for all students, not just the serious musicians/artists," Gitomer said. "These efforts, as well as many others, were focused on teachers and classrooms, rather than measures of accountability."
Even as several southern governors, including Richard Riley of South Carolina, James Hunt of North Carolina and Bill Clinton of Arkansas, began in the 1980s the standards movement that would become the strongest threat to portfolio assessment, some states experimented with portfolios on a large scale. Vermont and Kentucky investigated the possibility of using portfolio assessments instead of standardized tests to judge school, district and state progress in educational achievement.
But in 1994 RAND corporation researcher Daniel Koretz, now at the Harvard University Graduate School of Education, released a report on portfolio assessment in Vermont that many experts say marked the beginning of the decline of that form of grading students. Koretz said Vermont's assessment of student work suffered because one school might require one kind of project, and another school quite a different one. It was difficult to compare their work. Teachers, the Koretz report said, also complained that portfolios were cutting into valuable teaching time. Math teachers, he said, "frequently noted that portfolio activities take time away from basic skills and computation, which still needs attention."
Koretz's careful methodology and national reputation had an impact, but there were signs that portfolios were already losing ground. About the same time British Prime Minister John Major discarded the portfolio system that had been used for 20 years as the school-leaving English examination in Great Britain. Dylan Wiliam, a British assessment expert who now works for ETS, said Major felt "that timed written examinations were the fairest way to assess achievement at the end of compulsory schooling," although about 40 percent of the English exam grade and 20 percent of the math grade is still based on portfolio-like elements.
The decline of portfolios as a large scale accountability measure is not necessarily a bad thing, Gitomer said. "The power of portfolios resides in its coming out of the student's own classroom practice. The value resides in the wealth of information available and the various conversations that one can have about the work and the portfolio creator. If all you're going to do is give a single score, there's far more efficient ways of getting at a student's achievement level."
Ronald Wolk, founder of the newspaper Education Week, said he appreciates the need for large scale assessments, but thinks the standardized tests that are replacing portfolios are no easier to judge than actual student work. "Officials object to using portfolios for assessment because they are too subjective," said Wolk, who admired the Beacon School grading system. "But that is exactly how the writing on Regents' exams is scored. Teachers read and grade the exams according to their best judgment. At Beacon, at least, the teachers use rubrics that they have crafted and honed over the years."
Advocates reject the notion that portfolios work only if a school system embraces a model of education where students are not expected to master a broad range of material. Neill of Fair Test said the idea is to collect key pieces that provide evidence of learning in key areas. Even with a standard high school grading system, he said, "unless all children take precisely the same curriculum, and master it to a similar degree, and then recall it all, they will come into any college course with different aspects of knowledge and different gaps in that knowledge."
At Beacon, Stoll said the faculty is trying to maintain the portfolio system in a limited form, "but it is hard. You have the teacher telling the student to get his portfolio done and he says that he is studying for the regents test. It is like mixing two different currencies, and the bad currency drives out the good currency in a certain sense." Beacon's request to be exempt from the regents tests was turned down by Richard P. Mills, New York's commissioner of education, who had tried portfolio assessments, unsuccessfully, when he was the state school superintendent in Vermont.
The argument between advocates of standardized tests and advocates of portfolios usually ends with each side saying they cannot trust the results produced by the other. Authentic assessment "is costly indeed, and slow and cumbersome," said Chester E. Finn Jr., president of the Thomas B. Fordham Foundation and a supporter of standardized testing, "but I think it's biggest flaw as an external assessment is its subjectivity and unreliability."
Robert Holland, a senior fellow in the Lexington Institute, a think tank in Arlington, Va., raised the issue of cheating. "Scorers may have no way to tell if the work samples came from a student or a smart uncle or from an Internet download," he said. Portfolio supporters note that regular tests have also produced cheating incidents.
Lisa Graham Keegan, chief executive officer of the Washington-based Education Leaders Council, said she thinks portfolios can help teachers assess their students' progress, but are not a good tool for determining how a school or a district is doing. She remembers a visit to a northern Arizona school where "the writing teacher was showing me a portfolio of a student's work in which the student was writing about kamikaze pilots during World War II." Keegan was state school superintendent for Arizona at the time and saw that "the essay was horribly written with glaring spelling and grammatical errors and yet had received a score of 23 out of 25 points."
"The teacher was just glowing with what a mature and moving topic the student had chosen without any direction from her. I was less impressed and said so -- something along the lines of how I could appreciate that the student had something interesting to say, but my first impression was that he didn't know how to say it -- and wasn't that the first order task for the teacher?"
Having students display their personal strengths is fine, Keegan said, as long as they still learn to read, write and do math capably before they graduate. "A collection of student work can be incredibly valuable," she said, "but it cannot replace an objective and systematic diagnostic program. Hopefully, we will come to a place where we incorporate both."
A version of this column by Jay Mathews appeared in the Summer 2004 issue of the journal Education Next.