Ranking Ratings
This site is devoted more to analyzing computer rating systems than analyzing teams, and it would not be possible for me to do so without the data provided by Kenneth Massey's rating comparison page. Not only does Dr. Massey provide the raw material for any rating analysis, his and Dave Wilson's categorizations of systems provide the insight we need to analyze ratings and ideas about how to present the analysis.
As Dr. Massey says
It is a challenge to present so much information clearly on one web page. As the number of rankings has grown, a convienent display of them has become more difficult. Because it seems unneccessary to list the actual continuous scale ratings, I list only the ordinal rankings (i.e. 1st, 2nd, 3rd,...). This provides sufficient information to accomplish the goal of comparing teams and rating systems.
The plain text version sorts the teams by consensus ranking vertically, and sorts the ranking systems by correlation horizontally. Team names are listed at regular intervals so that they will always be visible. The high (red) and low (blue) rankings for each team are highlighted.
Dr. Massey displays the ratings left-to-right by how well they correlate to the average rating, and reports the retrodictive ranking violations for each rating (see the bottom of his page.)
I've produced a page that provides the same data in a different format to make visible some of the differences in the rankings.
- Ratings
- are listed left-to-right in increasing order of Weighted Retrodictive Ranking Violations.
|
WRRV = ⌈ (WS - LS)÷S ⌉ × (WR - LR) × ƒ(LR) |
- WS
- Winner's score
- LS
- Loser's score
- ⌈ (WS - LS)÷S ⌉ =
-
the margin of victory in number of scores
(⌈x⌉ is the least integer ≥ x)
S=8 for football, 3 for basketball.
- WR
- Winner's rank
- LR
- Loser's rank
Note that (WR-LR) is positive for all ranking violations by definition.
- ƒ(LR) =
-
Log LRi Mj Mi+j
where i and j are chosen to make ƒ(1) any predetermined value
ƒ(M) is exactly 1.
M is the number of ranked teams (120 for the Football Bowl Subdivision, 343 for D-1 basketball.)
|
See WRRV 2.1 for graphs of ƒ(LR) for different choices of i and j.
- Teams
- are listed in "Bucklin Majority" (BMaj) order.
When there is an odd number of computer rankings this is the same as the arithmetical median computer rank. When there is an even number of computer rankings BMaj is the higher of the two ranks used to form the arithmetical median.
- Rankings
- are color coded to indicate whether they are better than the BMaj rank (blue), equal to it (black) or worse (red.)