The Madness within the Madness: College Basketball’s Flawed Ranking Methodology

Ok, full disclosure: I had a great bracket for two days, then it all went horribly, horribly awry.  So yes, I am a little bitter.  But that doesn’t change the facts.  There’s a ranking problem in college basketball.

Determining the top 68 teams out of the 351 total teams in Division 1 is in itself daunting.  Then, after this initial decision, a committee must seed all of these teams.  Naturally, there are teams who should have made it but didn’t (“snubs”) and teams that should have been ranked higher or lower.  Of course, the ranking will never be perfect; some teams play harder schedules, peak later, get luckier, or experience any number of other things.  Yet there is a deeper issue with the ranking criteria.

One reason why the SAT has survived despite its less than stellar reviews is that it provides an independent standard by which to judge students.  College basketball, however, doesn’t enjoy the luxury of such an independent standard.  Instead, it must rely on statistics like strength of schedule and RPI.  These formulas attempt to illuminate how good a team’s record actually is.  A team with 5 losses but a very good RPI, for instance, should be ranked higher than a team with 3 losses but a significantly lower RPI.

There is an issue here though.  Without an independent absolute standard, each team’s status is determined by the status of other teams.  But each other team’s status is also determined by the status of other teams.  The formula for RPI is (WP * 0.25) + (OWP * 0.50) + (OOWP * 0.25), where WP is winning percentage, OWP is opponents’ WP, and OOWP is opponents’ opponents’ WP.  But why stop with OOWP? Why not OOOWP?  Or OOOOWP?  And so on.  There seems to be a sort of infinite regress here.  One might argue that we have more advanced metrics now, but they are still susceptible to the same issue.  Unless every team could play every other team, we’re kind of stuck (and even then, there are also other relevant factors. The outcome would still not be perfect).

In short, don’t trust the rankings.  While many factors (e.g. matchups, dumb luck, etc.) contribute to the consistently large number of upsets in March Madness, the flawed ranking methodology doesn’t help.  These other factors—exacerbated by the small sample size—ensure that even the NCAA Tournament doesn’t determine the best team correctly.  Yet we still watch it every year.  Because that’s not what we want from the NCAA Division 1 Basketball Tournament.  We don’t want it to determine the statistically best basketball team; we crave the madness.