In my last article I suggested a method for counting votes in the human poll that would provide the desired "transparency" of the polling process and still provide for anonymous ballots. In this one I'll turn my attention to how the polls (and computer rankings) might be used to avoid the problems with putting too much emphasis on one component.
The problem with human polls is that they are subject to human nature. A team that has a bye week is likely to fall in such a ranking, a team that's ranked highly early tends to remain that way even if other teams are playing better, and so on. But because human nature is somewhat universal, all of the human polls are subject to these phenomena. So, the first recommendation is
|Coaches and Harris Poll Comparison|
|Variance from Best||0.49||0.57|
After the first half of the season (really well before that) there's just not enough difference between any of the human polls to tell them apart. Given that the AP pulled out because their voting is too transparent and the coaches have an obvious conflict of interest or two, we may as well use the Harris poll. It is sponsored by the BCS anyway, and if humans should have more weight (with which I disagree) that can be factored into the formula.
As you can see from this comparison of the polls after week eight, the coaches poll and Harris poll:
The only difference in the variance from the better rank of the two polls is exactly that one has Virginia Tech 16th and the other 18th.
The computers are not nearly as unanimous, which in general is a good thing. They all have different means of handling strength of schedule, some have different weights for later games, some (like human voters) carry over results from prior years, some factor in game location, some opponents'and opponents' opponents' records, and so forth. To the extent that these are all important to judging the quality of a team, some synthesis of different perspectives is desirable.
I have always thought that the way the human polls and computer rankings were handled should be the same, and in 2004 they changed the formula so that it superficially was. However, the normalization was to the "# of voters" level, and still resulted in different weights for each "voter". Also, the "don't include best and worst computer ranking" has no analogue in the human polls (though Harris' "trimming" in their second poll was similar in spirit).
So this brings us to my second recommendation:
Taking only the ordinal ranks makes sense because we do not know in general how the different computers come up with theirs. The fact that we know the humans use a flawed election method (Borda) to do so does not make that method a useful one for combining the computer rankings into one. If we list only the ordinal rankings from each source component for week 9 we get:
BCS Computer Rankings + Harris Results|
(Top 25 only)
|Components are listed alphabetically left to right,|
and teams ordered only by leftmost rank.
There usually will be ties, though, and there's a two-stage tiebreaker. For teams with the same "majority ranking":
|Team||Majority||#≤Maj|| Tie |
|5||Penn State||5||4||787||Using Borda would've had UCLA ahead of Penn State even though a majority of the inputs had the Nittany Lions ranked higher than the Bruins|
|8||Ohio State||9||5||767||5 inputs have the Buckeyes ranked 9th or better but only 4 have the Hurricanes that high. This illustrates a stage 1 tiebreaker.|
|10||Oregon||11||4||1.51||655||Here we have an example of a stage 2 tiebreaker. Oregon has votes for #12 and #13, Georgia for #14, #17, and #18, LSU #15, #16, #17, and Texas Tech has 2 #16s and a #21. Clearly Oregon's closer to a 5th #11 vote (by far) than any of the others.
Borda would've ranked Oregon behind these teams, just because one input left them out.
Teams that aren't ranked in the top 25 by at least four of the seven inputs are listed in Borda order. But by using a "true" Borda count we can tell that the first four of these were listed on two of the inputs and Boise State on only one of the seven.|
So we know that that Auburn needs to move into the top 25 in two more computers to get a ranking, and Boise needs to improve in three.
This approach is very simple, and likely to result in better orderings than the current one which gives too much weight to the polls. Based upon the reaction a few years ago when the computers determined the best matchup, one would expect that there would be complaints that the human polls only have 1/7th input. That is not quite correct, since the influence that any one component has depends upon where it fits with respect to all of the other components.
In any case, for years we've avoided the real issue - if computers "aren't to be trusted" then we need some unambiguous way to define what "trusted" means in terms of picking the best two teams. That will be the subject of my next essay.