By Mark Phillips
There’s a story about a guy who looks out his window one night and sees a neighbor crawling around under the corner streetlight. He asks him what he’s doing. “I’m looking for a $10 bill I dropped,” replies the man. “Where did you drop it?” asks the neighbor. “Up the street.” Amazed, the neighbor asks, “Then why are you looking for it here?” “Because the light is better,” the man replies.
The story is a good metaphor for how we evaluate school performance. The most important data regarding school performance is lying up the street in the dark, while using standardized test scores is like looking under the streetlight. At least the man in the story is not under the illusion that a few coins he finds under the streetlight are the same as the $10 bill. And at least he knows what he’s lost. That’s more than can be said about how we usually evaluate school performance.
There is a common assumption that if a school has high test scores, a high admissions rate to good colleges, and a low dropout rate, it’s doing a good job. These are the easiest ways to measure school performance, so of course, that’s how we do it.
These are inarguably valid criteria in assessing school performance. The problem is that there are other important criteria that are not being used and may not reinforce the conclusions drawn from the conventional assessment. These, however, are not easy to apply. But if we don’t consider these other criteria, we may arrive at conclusions that are invalid and that adversely affect our educational program decisions. And solely using the easy criteria to determine which are our “distinguished” schools is foolish.
So what are these criteria that lie up the street in the dark?
Let’s begin with the Japanese hikikomori (a phenomenon I referred to in this column). This generation of young people in Japan, trained at high-achieving high schools and most accepted at good universities, eventually dropped out. Many are living with their parents, dissatisfied with their jobs, and alienated from society.
One lesson we can learn from this is that unless we do extensive follow-up studies of high school and college graduates, we have very limited information on how good a job we’re doing.
The first focus should be on college and job success and satisfaction immediately after graduation. This is critical in providing feedback for every high school. As just one example, the major study, Diplomas and Dropouts , a project of the American Enterprise Institute, found that fewer than 60% of new students graduated from four-year colleges within six years.
How well prepared are students from your high school for college? What is their college completion rate? For those not continuing their education, what is the employment rate? What are the emotional and social problems they encountered during the immediate post high school years? How effectively did the school help develop both the technological and social skills that most employers look for today?
There should also be a 10-year follow-up study that includes self-reports on employment rates, job success and satisfaction, relational success and satisfaction, and engagement in the socio-political world. This is all critical data.
Every school district should have at least one person assigned the primary task of conducting these follow-up studies. And there isn’t a single school district that can validly argue that there is no wasted administrative time that couldn’t be better used for this purpose. It should be required of every school district.
Of course, our schools aren’t fully responsible for what happens to students later, but schools that scored well when assessed by all of these criteria would be far more credibly designated as high achieving than those based on our present criteria. And the data would be extraordinarily helpful in formulating changes in school curriculum and instruction.
This should also serve as a cautionary warning not to congratulate ourselves on our excellence as we sit under the streetlight counting a few coins.
Follow The Answer Sheet every day by bookmarking http://www.washingtonpost.com/blogs/answer-sheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page. Bookmark it!