NCAA tournament 2012: Is it time to say RIP to the RPI?
By Eric Prisbell,
Since 1981, the NCAA tournament selection committee has used the Ratings Percentage Index as its mathematical formula to evaluate teams, and for the entirety of its existence, it has been assailed by fans and coaches whose schools did not fare well by its metrics. But now, after more than three decades of influence, its rivals are gaining popularity and its critics are growing louder, sending a blunt message to the three most controversial, misunderstood letters in college basketball:
The debate over the RPI’s use rages with heightened volume this season because there have never been more evaluation tools accessible to the general public, much less to the 10-member committee soon to be sequestered in an Indianapolis hotel.
Unlike when the RPI was introduced in the early 1980s, games flood television airwaves and statistics are readily available at the click of a mouse, providing countless opportunities to assess teams from leagues famous and obscure. And in a metric-crazed era that former Maryland coach Gary Williams once deemed the “Revenge of the Nerds,” more sophisticated ratings systems, in the eyes of many, have rendered the RPI antiquated and insufficient.
No one painted a more visual image of the RPI as a relic than Scott Van Pelt, the ESPN personality who, in calling it the worst metric in sports during a seven-minute radio rant, likened the reliance on the RPI to a man “walking around with a big Walkman on his hip the size of his toaster, who is flipping over his cassette tape, who wants to run home to program his VCR on his standard-definition television.”
The NCAA’s view of the RPI is conflicted. While examining potential NCAA tournament teams, selection committee members study team profiles that are broken down by victories over specific sub-groups of the RPI, such as wins over teams ranked in the top 50. At the same time, officials attempt to distance themselves from the RPI, saying they don’t rely on a team’s individual rating or a conference’s RPI and that they consider a handful of metrics.
What’s more, Jeff Hathaway, the selection committee chairman, said that how a team appears to the eye is “crucial,” adding that “we need to go beyond the numbers.”
The RPI formula, tweaked over the years, considers a team’s winning percentage (25 percent), its opponents’ winning percentage (50 percent) and its opponents’ opponents’ winning percentage (25 percent). Because of concerns over coaches potentially running up scores, it does not consider margin of victory.
For instance, Wichita State earned a 21-point win at Creighton on Feb. 11, one of the more impressive road victories by any team this season. On the heels of the performance, the Shockers entered the Associated Press top 25. But in the RPI, their position did not significantly improve.
Yet the RPI remains relevant because the NCAA deems it relevant. John Gasaway, an ESPN.com writer who this month explored the birth of the RPI in a 4,100-word story for Basketball Prospectus, said: “It draws its power exclusively from its sponsor. People who defend the use of the RPI do not themselves use the RPI to make points during the season. You don’t read a column saying, ‘Hey, everybody look at Southern Miss. They are a good team, and the reason you can tell is because they have an RPI in the top 15.’ ”
‘I don’t like the RPI’
The sentiment of some rating gurus is that the RPI should be used as nothing more than a rough tool. Joel Sokol, an engineering professor at Georgia Tech whose ratings system (LRMC) is used by the selection committee, said the RPI is “not even close to the best indicator of a team’s strength.”
But the man who has done the most to kill the RPI is — of all people — a 38-year-old Utah-based meteorologist at the National Weather Service. While growing up in Alexandria, Ken Pomeroy would ask his mother to buy the Tuesday edition of USA Today so he could study the weekly ratings of Jeff Sagarin. Pomeroy yearned for the formula.
Two decades later, Pomeroy became the most prominent proponent of tempo-free statistics, where a team’s strengths are measured possession by possession, rather than using traditional measurements such as points scored per game. Close to 100 Division I schools subscribe to his Web site (KenPom.com). He has appeared on Duke Coach Mike Krzyzewski’s radio show.
One basketball insider likened Pomeroy to Republican presidential candidate Ron Paul because of the unabated passion displayed by supporters, a comparison Pomeroy understands. Pomeroy’s fans are so invested that even when Pomeroy acknowledges an outlier in his ratings, he will receive e-mails screaming: “No! You have to trust the numbers!”
Pomeroy used to simulate RPI on his Web site until he got a call from a current Southeastern Conference coach lambasting him for his team’s position in what the coach said was Pomeroy’s RPI. Now, Pomeroy said he looks at the RPI exactly zero times during the season. And he has a word for anyone who relies solely on it to make judgments about teams: lazy.
“I don’t like the RPI,” Pomeroy said. “It has no analytical value whatsoever.”
‘It’s hard to kill’
One person who takes issue with some emerging metrics is Jerry Palm, a defender of limited uses of the RPI who has gained popularity this past decade for simulating the RPI on his subscription Web site (CollegeRPI.com) and projecting the tournament field, most recently for CBSSports.com.
Not a week passes without someone incorrectly assuming Palm created the RPI. He has been introduced on radio shows as its inventor and asked how often he tweaks his formula. In recent years, Palm has helped about a dozen schools or conferences with scheduling concepts intended to improve RPIs.
When asked if it’s time to replace the RPI with other metrics, Palm said: “So what are their formulas? Really, what are the exact formulas? They are secret. So we don’t have any idea how these teams are being judged other than that we take their word for what is going into it. At least the RPI can be explained.”
With a different metric, Palm said the committee would be expected to almost blindly follow ratings, forcing members to say: “ ‘We had to do this because Pomeroy says whatever.’ You don’t really want computers to make these decisions.”
Palm said too many people underestimate how thorough a job the committee does every year, and even the other rating gurus acknowledge the bracket would not look dramatically different if the RPI were tossed aside. But is there another alternative?
Gasaway said that one NCAA response to “these guys clamoring at your door with their laptops and advanced systems would be: ‘Okay, smart guy, rock my world. Design me a system that measures 100 percent team quality and zero percent coach swinishness of trying to run up the score, and we’ll use them.’ ”
Tennessee-based Kenneth Massey, whose ratings are also considered by the committee, said the RPI would quickly “drop into a footnote in the history books” if no longer used by the NCAA. But Sokol, the engineering professor, disagreed. In the years that followed, he said, anyone with a calculator and a Web site could sweep off the cobwebs and resuscitate the three-letter artifact that’s as old as MTV.
Sokol paused, then concluded, “It’s hard to kill.”
Related: Four alternatives to the RPI