Higher education leaders often call college rankings misleading. The rankers, they say, insert questionable data on schools into a subjective formula and produce numbered listings of “top colleges” that have only a veneer of validity and objectivity.
Students value selectivity in admissions, and strong academics, and research prowess. They want a quality education at a good price, that will help them earn a good salary when they leave school, and not get stuck in debt. They admire schools that work for the public good and serve as an engine of social mobility for the disadvantaged to reach the middle class.
So why not take several rankings that seek to capture all of those qualities and combine them into one?
The concept is no doubt debatable for a host of reasons. Crunching ordinal numbers derived from other numbers is a tricky proposition and certainly statistically dubious. But with a plethora of rankings emerging in recent years, each with its own formula, there is an opportunity to average the rankings and come up with a composite that takes them all into account.
Here’s the experiment.
Start with two lists from U.S. News and World Report: national universities and national liberal arts colleges. These rankings, which are based in part on selectivity, wealth and reputational surveys, for decades have been the most prominent in the market.
For the national universities, add:
- National university rankings from Washington Monthly, which aim to gauge their contribution to the public good.
- Rankings from the Wall Street Journal/Times Higher Education, which include a focus on outcomes such as graduation rates, salaries and student engagement.
- Rankings from Times Higher Education world university analysts, only of schools within the United States, which focus on research prowess.
- Rankings from Money and Forbes magazines, which in different ways seek to measure value and outcomes. Both include data on salaries of alumni.
Take the sum and divide by six (the number of rankings in play). Order the results from low to high. Assign them each a new rank, from 1st to 121st. Here is what you get:
Combined 2016 university ranking:
Below is a similar exercise with 120 liberal arts colleges. Except in this case, there is no global rank for research from Times Higher Education. So here, the formula averages five separate rankings, with the result re-ranked.
Combined 2016 liberal arts college ranking:
There are plenty of caveats to all this. Significantly, the Washington Monthly and U.S. News rankings are composed of separate lists of universities and liberal arts colleges. The other rankings are combined into one list. So that poses an obvious challenge for comparisons. Williams College, for instance, ranks 22nd on the Journal/THE combined list, 49th on the Money combined list and 1st on the U.S. News liberal arts list.
The goal of these composite rankings is to use the top-line numbers for each school that are presented to consumers through the various rankings. And Williams doesn’t appear to suffer much through the combined formula. It comes out third.
In general, the usual suspects do well in all the rankings. Those known as Top 10 schools are basically Top 10 across the board, with some re-arranging: Stanford, Harvard, MIT and the like.
But in the middle of the combined rankings are some surprises: Schools that fare better in the average than they do solely through U.S. News. An example is the University of Maryland at College Park.
With 38,000 students, U-Md. is known as a research powerhouse. U.S. News ranks it 60th among national universities, public and private, a position that it has held, with slight variations up and down, for several years. That is certainly a prominent rank. The Journal/THE list places U-Md. at 100th.
But in the combined list, boosted by high marks from Money and Times Higher Ed global, College Park ranks 37th, ahead of Wake Forest, Carnegie Mellon, Boston College, New York University and several other schools that U.S. News esteems.
“Rankings matter, period,” said U-Md. President Wallace D. Loh. “That said, are they valid? They’re valid in a very general sense. When you take a large, complex organization and reduce it to one number, it simply doesn’t capture it all.”
Loh said the university strives for quality and affordability, and can show results on both fronts. Tuition and fees at College Park total about $10,200 for Maryland students, not counting room and board. That compares very favorably to private schools and many public institutions.
“Once you take affordability into account, we zoom in the rankings,” Loh said.