Conference Ranks

© Copyright 2011, Paul Kislanko

There are as many ways to compare conference strength as their are opinons. Simply averaging teams' ratings (or ranks) doesn't sufficiently account for the fact that a conference may contain one very good team or a few very bad teams that skew the average.

Here are a couple of methods I find most useful, along with the reasons why I do.

Team Ranks

I actually use a "weighted median" - which is a bad name because it's really a modified average. The adjustment takes into account how much better/worse the teams' ranks are than the field median. It gives a rough measure of how hard it is to have a good conference record in a given conference.

Rank Distribution by Conference^{Note 1}
17 Oct 2011

I'm not fond of any meta-ranking that depends upon rankiings, but this one does make an easy distinction between AQ-conferences and the dregs of D1. Since this is based upon ordinal rankings, lower scores are better.

Ratings, not Rankings

Rating Distribution by Conference^{Note 2}
17 Oct 2011

While the weighted median gives more weight to the middle strength of the conference, this "grade summary" emphasizes how good the top teams (in the field) are. The sort sequence gives 256 points for each team whos rating is more than 2 standard deviations better than average, 128 for teams with ratings less than 2 but more than 1.5 SDs better, down to 0 for teams whose ratings are more than 2 standard deviations below average. The sort value is the sum of all those divided by the number of teams.

Averages on Steroids

A problem with each of the aggregations I've talked about is that the weights ultimately depend upon the number of teams in the conference and those aren't the same for every conference. It's a little more work to calculate, but there is a lot to be said for what I call the Team-Pairwise Conference Comparison.

The basic idea is embodied in the aggregation's name - compare each team from conference A to every team in every other conference based upon some particular rating/ranking. For example, to compare the Big 12's ten teams to the SEC's 12 teams requires 60 comparisons (12×10)/2.) If we are comparing teams' ranks the resulting table gives the probability that if that you choose a team at random from the conference listed in the row and a team at random from the conference listed in the column heading the rank is better for the row-conference than the column-conference.

Team-Pairwise Conference Comparison
Probability Rank(team from row-conf) < Rank(team from column-conf)

This conference rating (I sorted by the percentage of pairwise-comparisons in which the conference's teams came out better) including the conference-pairwise results is more useful than other "conference rankings based upon ratings" because it easier to see how even though conference B is lower on the list than conference A, it still might be ahead of conference A if they were the only two conferences. For example, the ACC does better against the entire field than the Big 10, but the Big 10 does better against the ACC than vice versa. (There's a math lesson here but I'll save that for a different audience.)

There's more to like about this method. If you save the intermediate results for each team you get a rough approximation of how each team would compare to teams in conferences other than its own. This is highly useful in an era when lots of teams are considering changing conferences.

Notes

1. All of the examples in this essay are based upon Jeff Sagarin's "composite" rating as of 16 October 2011. The rating was chosen because the data is public and includes sufficient detail to illustrate each of the methods included in the article.
2. μ is just the aritmentical average of all teams' ratings. σ is the standard deviation. The chart just places each team in a ½σ-wide bucket based upon its rating.