The 2015 “World University” rankings were released by Times Higher Education in London on Wednesday, but don’t confuse them with the “2015 World University Rankings” issued in July by a Saudi Arabian group, or the U.S. News & World Report’s “2015 Best Global University Rankings,” or the 2015 “Academic Ranking of World Universities” put out in August by the Center for World-Class Universities at Shanghai Jiao Tong University in China, or … well, you get the idea. Be very, very careful with school rankings.

And to answer the question you are about to ask: No, the results aren’t identical.

Harvard is No. 1 on three of the four rankings mentioned above — all but the Times Higher Education list, which gives the top spot to the California Institute of Technology, for the fifth straight year. Harvard ranks sixth on the Times list. Caltech is seventh on the list from China as well as on the U.S. News list but 12th on the list from Saudi Arabia. The University of Oxford in England is No. 2 on the Times Higher Education list, but No. 5 on the list from Saudi Arabia as well as U.S. News, and No. 10 on the list from China.

Each list has its own methodology and its own criteria, though there is overlap. Naturally, each claim to be extra-special. For example, The Times Higher Education Web site says:

The Times Higher Education World University Rankings 2014-2015 list the best global universities and are the only international university performance tables to judge world class universities across all of their core missions – teaching, research, knowledge transfer and international outlook.

And the Web site of the Academic Ranking of World University says:

Starting from 2003, ARWU has been presenting the world Top 500 universities annually based on a set of objective indicators and third-party data. ARWU has been recognized as the precursor of global university rankings and the most trustworthy league table.

Let’s look at the release of the Times Higher Education World University Rankings 2014-15, which offers good and bad news for the United States. The good news: 147 U.S. institutions are on the list of 800 schools.  That’s far more than Britain, with 78; Japan, with 41; China, with 37; and Germany, also with 37. What’s more the California Institute of Technology is No. 1 for the fifth straight year. The bad news: The U.S. dominance may be threatened in “years to come.” (As bad news goes, that’s not so bad.)

In the release, Phil Baty, editor of the Times Higher Education World University Rankings, said:

“The United States still dominates the World University Rankings, with a staggering 147 institutions featured in this prestigious list, including California Institute of Technology, which has held on to pole position for the fifth year in a row. Despite this achievement, the US will have to watch out for countries in continental Europe, such as Germany and the Netherlands, and Asia, such as China, that may threaten its status in years to come. While 47 states in the US have implemented higher education funding cuts since the recession in 2008, including California, which cut its university system’s budget by $900 million, or 27 per cent, these standout performers in Europe and Asia are continuing to invest heavily in higher education. The US will have to raise its game to ensure its dominance does not erode.”

Note: Despite his warning about California’s disinvestment in higher education, which is real and unacceptable, it turns out that the U.S. News list of world-class universities includes three public schools from California — the University of California at Berkeley at No. 3, the University of California at Los Angeles is No. 8 and the University of California at San Diego is No. 18.  Meanwhile, the University of Michigan is in a tie for No. 14 with two other schools, including the University of Washington. That makes a total of five public U.S. universities on the U.S. News list — one fourth of the top 20.

Here are the top 20 schools in THE TIMES HIGHER EDUCATION WORLD UNIVERSITY RANKINGS 2015-16:

Rank 2015-16 Institution Country
1 California Institute of Technology US
2 University of Oxford UK
3 Stanford University US
4 University of Cambridge UK
5 Massachusetts Institute of Technology US
6 Harvard University US
7 Princeton University US
8 Imperial College London UK
9 ETH Zürich – Swiss Federal Institute of Technology Zurich Switzerland
10 University of Chicago US
11 Johns Hopkins University US
12 Yale University US
13 University of California, Berkeley US
14 University College London UK
15 Columbia University US
16 University of California, Los Angeles US
17 University of Pennsylvania US
18 Cornell University US
19 University of Toronto Canada
20 Duke University US

How was this list created? Thirteen performance indicators are grouped into five areas:

  • Teaching: the learning environment (worth 30 percent of the overall ranking score)
  • Research: volume, income and reputation (worth 30 percent)
  • Citations: research influence (worth 30 percent)
  • Industry income: innovation (worth 2.5 percent)
  • International outlook: staff, students and research (worth 7.5 percent).

As you can see, reputation matters, as does money. Measuring reputation can be tricky, it is worth noting, giving that it is entirely subjective and all of these listings claim to be using objective measures based on solid data.

Here are the 13 performance indicators:

Industry income – innovation
1. Research income from industry/academic staff. Yes, income matters here.
Teaching – the learning environment
2   Reputation survey – teaching. Yes, reputation matters, as if reputation always matches reality.
3.  Staff-to-student ratio
4.  PhDs/undergraduate degrees awarded
5.  PhDs awarded/academic staff
6.  Institutional income/academic staff
Citations – research influence
7.  Citation impact (normalized average citations per paper)
Research – volume, income and reputation
8.  Reputation survey – research
9.  Research income/academic staff
10. Scholarly papers/academic staff and research staff
International outlook – staff, students and research
11.   International students/total students
12.   International academic staff/total academic staff
13.   Scholarly papers with one or more international co-authors/total scholarly papers

The Academic Ranking of World Universities — which includes a basic list and then sub-lists based on subject — is conducted by researchers at the Center for World-Class Universities of Shanghai Jiao Tong University in China, which has been focusing on the study of world-class universities for many years,  hosting the “First International Conference on World-Class Universities” in 2005 and organizes the conference every second year. The center also builds databases of major research universities in the world and clearinghouse of literature on world-class universities, and provide consultation for governments and universities.

Here are the top ARWU 20 on its rankings of 500 universities:

1. Harvard University
2. Stanford University
3. Massachusetts Institute of Technology (MIT)
4. University of California, Berkeley
5. University of Cambridge
6. Princeton University
7. California Institute of Technology
8. Columbia University
9. University of Chicago
10.University of Oxford
11. Yale University
12. University of California, Los Angeles
13. Cornell University
14. University of California, San Diego
15. University of Washington
16. Johns Hopkins University
17. University of Pennsylvania
18. University College London
18. University of California, San Francisco
20. Swiss Federal Institute of Technology Zurich

The Academic Ranking of World Universities, the Web site says, “adopts six objective indicators to rank world universities, including the number of alumni and staff winning Nobel Prizes and Fields Medals, the number of Highly Cited Researchers, the number of articles published in journals of Nature and Science, the number of articles indexed in Science Citation Index – Expanded and Social Sciences Citation Index, and per-capita performance. More than 1,200 universities are actually ranked by ARWU every year and the best 500 universities are published.”

The king of college rankings, U.S. News & World Report, got into the world ranking business in October 2014, with its “2015 Best Global University Rankings.” Here’s the U.S. News list:

1. Harvard University
2. Massachusetts Institute of Technology
3. University of California–Berkeley
4. Stanford University
5. University of Oxford
6. University of Cambridge
7. California Institute of Technology
8. University of California–Los Angeles
9. University of Chicago
10. Columbia University
11. Johns Hopkins University
12. Imperial College London
13. Princeton University
14. University of Michigan
14. University of Toronto
14. University of Washington
17. Yale University
18. University of California–San Diego
19. University of Pennsylvania
20. Duke University

U.S. News is famous for using reputation as a key criteria in its listings of U.S. colleges and universities — and it uses it, too, in the global university rankings, as the chief factor — 12.5 percent for global research reputation and 12.5 percent for regional research reputation. The rest of the criteria:

Publications 12.5%
Normalized citation impact   10%
Total citations  10%
Number of publications that are among the 10 percent most cited 12.5%
Percentage of total publications that are among the 10 percent most cited 10%
International collaboration 10%
Number of Ph.D.s awarded  5%
Number of Ph.D.s awarded per academic staff member  5%

This past July, a different list of the world’s best universities was released as an annual exercise by something called the Center for World University Rankings. What is the Center for World University Rankings? The Web site doesn’t exactly spell it out, but it is a Saudi-based consulting group that puts out university rankings, or, at least has done every year since 2012. The president of the center is Nadim Mahassen, an assistant professor at King Abdulaziz University in Jeddah, Saudi Arabia.

The center says it publishes “the only global university ranking that measures the quality of education and training of students as well as the prestige of the faculty members and the quality of their research without relying on surveys and university data submissions.”

Measuring prestige? Hmmm. Take a look at what the center calls “eight objective and robust indicators” that it uses to rank what it says are the world’s top 1000 universities:

1) Quality of Education, measured by the number of a university’s alumni who have won major international awards, prizes, and medals relative to the university’s size [25%]
2) Alumni Employment, measured by the number of a university’s alumni who have held CEO positions at the world’s top companies relative to the university’s size [25%]
3) Quality of Faculty, measured by the number of academics who have won major international awards, prizes, and medals [25%]
4) Publications, measured by the number of research papers appearing in reputable journals [5%]
5) Influence, measured by the number of research papers appearing in highly-influential journals [5%]
6) Citations, measured by the number of highly-cited research papers [5%]
7) Broad Impact, measured by the university’s h-index [5%] [which attempts to measure the quality and impact of a scholar of scientist’s body of work]
8) Patents, measured by the number of international patent filings [5%]

Apparently, being a great teacher to students doesn’t factor into the “quality of education.”

Here are the top 20, in order of ranking, as measured by the indicators above:

1. Harvard University
2. Stanford
3. MIT
4. University of Cambridge
5. University of Oxford
6. Columbia University
7. University of California, Berkeley
8. University of Chicago
9. Princeton University
10. Cornell University
11. Yale University
12. California Institute of Technology
13. University of Tokyo
14. University of Pennsylvania
15. University of California, Los Angeles
16. Johns Hopkins University
17. Kyoto University
18. New York University
19. University of Michigan, Ann Arbor
20. Swiss Federal Institute of Technology in Zurich

In these rankings, the United States had 229 schools ranked in the top 1,000, by far more than any other single country. China had 83; Japan, 74; the United Kingdom had 65; Germany, 55; France, 49; Italy, 47; Spain, 40; South Korea, 36; Canada, 33; Australia, 27. Russia had only five, fewer than Israel’s seven.

And the lesson in all of this? Once again, beware of school rankings.