The NCAA announced this summer that it would be replacing the Rating Percentage Index, used to help select and seed teams in the men’s NCAA tournament since 1981, as its primary tool for evaluating teams’ résumés. On the surface it seemed like a great idea: RPI was an inefficient means of gauging a team’s strength. Instead, the NCAA ushered in a new ranking system, the NCAA Evaluation Tool, or NET rankings. In doing so, the NCAA may have moved from bad to worse in the team metrics department. But it could have a highly entertaining effect on the NCAA tournament by prompting more upsets than usual.
RPI long had been viewed as a blunt tool for team evaluations that often produced poor results when it came to seeding. For example, RPI would have picked just 38 of 63 winners correctly in last year’s tournament. The early results of the NET rankings, which rely on “game results, strength of schedule, game location, scoring margin, net offensive and defensive efficiency and the quality of wins and losses,” don’t appear to be tracking much better. In fact, in some ways it is much, much worse.
I also think, philosophically, NET suffers from a "throw a bunch of metrics at the wall and see what sticks" problem, which is often characteristic of unthoughtful algorithm design, instead of having considered more deeply what it's actually trying to predict or achieve.
— Nate Silver (@NateSilver538) November 26, 2018
The first set of NET rankings, unveiled Nov. 26, had Ohio State, Virginia and Texas Tech as its top three teams. None of those squads were in the top three spots of the Associated Press’s poll that same week and only Virginia was in the AP top 10. That’s not to say one is right and the other is wrong, but consider Virginia was the consensus No. 4 team in the nation among 19 other ranking systems, with Ohio State (No. 13) and Texas Tech (No. 22) significantly lower than where the inaugural NET rating placed them that week. This week there were 31 methods surveyed (other than NET) causing even more disagreement over which schools should be ranked closer to the top. (RPI rankings are obsolete, but they are still tabulated and updated by Nolan Analytics and are featured here for reference.)
Virginia, per the NET rankings, is the No. 1 team in the country, but the Cavaliers have just one computer rating, developed by David Wilson, placing them in the top spot. Their consensus ranking, in the 31 power rankings monitored by the Massey Ratings, is No. 4; the Cavaliers would be No. 20 per RPI. Texas Tech also appears out of place, ranked as the fourth-best team per NET ranking but listed as the 10th best team per the consensus and the 25th best team per RPI.
NET ranking | School | RPI rank | Consensus rank of 31 other methods |
|---|---|---|---|
1 | Virginia (9-0) | 20 | 4 |
2 | Michigan (11-0) | 6 | 2 |
3 | Duke (9-1) | 10 | 1 |
4 | Texas Tech (10-0) | 25 | 10 |
5 | Kansas (9-0) | 1 | 3 |
6 | Tennessee (8-1) | 3 | 7 |
7 | Nevada (11-0) | 4 | 9 |
8 | Michigan State (8-2) | 9 | 5 |
9 | Wisconsin (9-2) | 8 | 14 |
10 | Houston (9-0) | 23 | 21 |
You might not think such a discrepancy in the rankings would mean much, but consider how this could affect the NCAA tournament, where a team such as Texas Tech would be given a No. 1 seed via its NET ranking even though it plays more like a No. 3 seed, per its consensus ranking.
While that would certainly benefit the Red Raiders by giving them an easier foe than if they were to face a No. 14 seed, it also means we’d be more likely to see another No. 1 seed fall to a No. 16, because Texas Tech would be much weaker than the teams we’ve historically seen seeded on the top line. If we forecast a top-four NET team earning a No. 1 seed in the tournament, their average win probability would be around 93 percent — which is actually a pessimistic forecast considering the No. 16 seed has a 1-135 record against the No. 1 seed since 1985, or a mere 0.74 percent upset rate. For reference, Virginia had a 97 percent win probability heading into its ill-fated matchup with UMBC last March.
Because of this dynamic in the NET rankings, where teams appear to be significantly over- and under-seeded, we would expect more surprises throughout the bracket once the tournament begins. An RPI in the top 15 accounted for 93 percent of the top-three seeded teams in the tournament since 2010, the first year the tournament expanded from 64 teams. If we assume the NCAA selection committee will simply swap out a team’s RPI rating for its NET ranking, it isn’t far-fetched to think Houston (10th in NET ranking, but 23rd in the consensus rankings), or a team in a similar situation, will be over-seeded in this year’s tournament, simultaneously giving an advantage to a lower-seeded team while robbing a better squad of its proper place in the bracket.
Not to pick on Houston, but it provides a strong example of the flaws of the NET rankings. The Cougars’ résumé, aside from an unblemished 10-0 record, isn’t spectacular in the eyes of many power rankings. After adjusting for strength of schedule, they are 14.2 points per game better than an average team, which places them 40th out of 353 schools. Ken Pomeroy’s rankings give them a similar slot (37th) after adjusting their points scored and allowed per 100 possessions (plus-15.4 adjusted efficiency margin).
If you were to slot Houston into a bracket based on its Pomeroy ranking — where the top four teams would earn No. 1 seeds, the next four No. 2, etc. — the Cougars would measure up as a No. 9 seed. But the NET rankings currently have them as a No. 3 seed, and if that’s the case, they would be significantly weaker than last year’s No. 3s: Michigan State (plus-25.4 adjusted efficiency margin, 6th per Pomeroy), Michigan (plus-24.2, 7th), Texas Tech (plus-22.6, 11th) and Tennessee (plus-22.3, 13th).
Houston’s efficiency margin is really nothing special. Baylor (plus-16.0) and Notre Dame (plus-15.8) had similar efficiency margins in 2017-18 to what Houston sports now and those two schools failed to qualify for the Big Dance.
This all could make top seeds more vulnerable than usual. Yes, teams are over- and under-seeded all the time in March, but based on last year’s consensus rankings, a top-four consensus team had an average RPI ranking of 3.3. This year the average NET ranking of a top-four team is 5.5, almost identical to a team ranked between No. 5 and No. 8 in the consensus group. In other words, the NET rankings are incapable of distinguishing between a No. 1 or No. 2 seed in the NCAA tournament, a stark contrast compared to last year where, via RPI, there was a clear difference between the two.
Given this new NET rating dynamic, there could be a lot more parity in the 2019 iteration of March Madness than there has been in years' past. And if the disparity between NET rankings and the consensus rankings holds up on Selection Sunday, you may want to be a little more liberal with upset picks in this year’s bracket pool.
Read more from The Post: