The University of Vermont came alive for Carly Lehrer, a visiting high school senior, during a lecture this month on environmental science. It was 8 a.m, and Lehrer was half asleep, but the room buzzed with energy and interest. That told her, in a way no college guide had, that Vermont was a school she would love to attend.
It was the kind of observation that college applicants, their parents and their high school counselors want to quantify, so they can compare the quality of teaching and learning on undergraduate campuses -- just as they compare SAT averages, graduation rates and many other indirect measures.
Until a few years ago, the task was impossible. But next month, an unusual organization based at the University of Indiana will celebrate the third anniversary of its effort to turn intellectual excitement and campus ferment into a set of statistics.
The National Survey of Student Engagement (NSSE, pronounced "Nessie") has collected more than 285,000 student surveys from 618 four-year colleges and universities. Even its for-profit counterpart, U.S. News & World Report's "America's Best Colleges" list, has begun to use some of the data, which is akin to Coca-Cola billboards displaying ads for Pepsi.
"NSSE is a window into areas of student and institutional performance that virtually all colleges and universities espouse to be important, but about which few have solid information," said George D. Kuh, the Indiana professor of higher education who directs the effort.
The surveys of undergraduate freshmen and seniors aim to measure a college's level of academic challenge, active and collaborative learning, student-faculty interaction, educational enrichment and supportive campus environment. Questions include: How many papers of five to 19 pages were you asked to write this year? How often did you discuss ideas with faculty outside of class? Did you study abroad? How do you rate the quality of your relationships with other students?
The third annual report, released this month, says that although 87 percent of undergraduates rated their college experience good or excellent, 26 percent of seniors reported that they never discussed ideas from their reading or classes with faculty members outside of class. Forty percent of undergraduates indicated they spend 10 or fewer hours a week preparing for class, much less time than their professors say is necessary.
Many high school students say they like this inside look at colleges, but they complain that it does not cover enough schools and that most participating colleges do not publicize their results.
When U.S. News asked colleges for some of their NSSE data this year, the magazine was able to publish information for only some schools -- 86 in the print version, 116 online -- because so many refused to make it public.
With some exceptions, such as the University of Virginia and Rice University, the better known and more selective a college is, the less likely it is to allow NSSE on its campus.
Alan Blickenstaff, a senior at Catalina Foothills High School in Tucson, examined the NSSE Web site last week and said, "I was disappointed. It had none of the eight colleges I am applying to -- Harvard, Dartmouth, Carleton, Claremont McKenna, Williams, Whitman, Pomona and the University of Arizona."
Some college officials say they remain uncertain about NSSE because it does not fit with their own assessment schemes. The schools say the data are useful in quietly identifying problems that need to be fixed. But they resist releasing the survey results because, although NSSE refuses to use the information to rank colleges, such publications as U.S. News might do so.
NSSE was born because many college officials said they felt U.S. News and other guides failed to measure the teaching and learning that is essential to higher education. Some pioneering researchers, such as C. Robert Pace of the University of California at Los Angeles, developed ways to ask students how well they were taught in classes and engaged in the rest of college life. Russell Edgerton, director of the Pew Forum on Undergraduate Learning, and Lee Shulman, president of the Carnegie Foundation for the Advancement of Teaching, pushed the idea of a national survey that would involve as many institutions as possible.
In December 1999, the Pew Charitable Trusts provided a $3.3 million grant to get the project started until it could be sustained by fees paid by colleges.
The first survey was conducted at 276 colleges and universities, and the effort is still growing, recently adding an Institute for Effective Educational Practice to help colleges and universities fix what the surveys show is broken.
"The NSSE database now includes information from institutions that represent more than half [52 percent] of all undergraduates attending four-year colleges and universities," Kuh said.
But it is a very mixed collection of schools. In the Washington region, the University of Virginia and the University of Maryland campuses at College Park, Baltimore County and the Eastern Shore participate, but Georgetown University and Johns Hopkins University do not.
Even those schools that have NSSE data and agree to release the information vary greatly in their efforts to make it easy to find.
Type "NSSE" into the University of Virginia Web site, steer your way to the "reports" link, and a long column of data appears. Virginia provides both good news (U-Va. students rate their educational experience far above national averages) and bad (U-Va. freshmen are not happy with academic adviser services).
George Mason University and Longwood University also provide substantial information, but only seven other local colleges provide such data -- and their Web site entries are hard to find or very brief.
Lehrer, a student at Sidwell Friends School in the District, said she was not familiar with NSSE but thinks the idea has promise. Following her recent visit to the University of Vermont, she applied. The university's provost, John Bramley, said the school plans to put its survey data on its Web site soon.
"That's the kind of information I need," Lehrer said. "If I had known about it, I would have looked at it."