Polls and Ratings

August 24, 2017

Dr. Massey tweeted on 22 August:
AP agrees with computers on 23/25 top teams. http://www.masseyratings.com/cf/compare.htm
giving me the opportunity to again rant about how misleading the results of the media polls can be because of the way they tabulate the votes.

118Ohio State22323
108Florida State33565
101Southern California44777
86Penn State66111110
67Oklahoma State1011141414
33Virginia Tech2122181818
21Kansas State2019252322
16Washington State24 232021
14South Florida1921 24 
11Texas A&M  242221
10West Virginia2220   
6Western Michigan  2125 
4Colorado  22  
2North Carolina    24
1Georgia Tech    25
1Utah 25   
First, the consensus ranking at College Football Ranking Composite isn't just the "computers", since it includes the human polls. Including them neglibily affects the composite by moving it in the direction of their rankings. My analysis only includes computer-based rankings that rank all division 1A teams, so (very slightly) represent "the computers" better.

Of more interest is that if we were to count the computer rankings the same way the AP does theirs, we'd get a different "top 25" than the average. This is because the computer average includes all the votes for #26 #27 #28 … and the "top 25" method ignores those. If we only consider the top 25 from each computer ballot and count the computers' "ballots" the same way the AP counts voters' ballots, we get a different "computer top 25."

Turns out the agreement with the AP is 22 of 25, and the content of the computers' top 25 has two changes. When you use the same vote-counting method for the computers as for the AP poll, Western Michigan and Colorado replace North Carolina and Georgia Tech in the "computer top 25." With the human-like truncated ballots, the computers like Texas A&M along with the Broncos and Buffs, while the AP (and coaches) like South Florida, West Virginia and Texas.

In the table to the right BORDA corresponds to the tally using all 130 ranks, equivalent to the consensus rank from the College Football Ranking Composite page with only computer ratings included. MIX is the top-25 as ordered by the median (16th-best) of the 31 computer ranks. CT25 is the pseudo-borda count using 25 points for each #1 rank, 24 for #2, and so on down to zero for ranks worse than 25th.

Note that MIX is not the median of the computer rankings. There are only 19 teams whose median rank is 25 or better, so the list is finished with six teams whose median ranks are 26-29. An up-to-date list of this ranking is maintained at Computer Rankings by Bucklin Majority. The updated version of the full BORDA report is available at Computer Rankings by Borda Count.

Meta is the pseudo-borda approach applied to the five rankings as if they were voters in the AP poll. It doesn't mean anything, but provides a convenient sort sequence for the report.

The human polls agree on 24 teams, with the AP including Washington State and the coaches liking Utah. One wonders if the AP voters were influenced by the earlier publication of the coaches poll.

There are finer measures of how similar rankings are than the number of teams in common. For each pair of teams included in either of two lists, call the pair concordant if they are in the same relative order in both lists, discordant if they are in the reverse order, and ignore the pair if they are tied in either list. These counts can be used to define rank correlation coefficients such as

Kendall's tau:
τ = #Concordant pairs - #Discordant pairs

Goodman and Kruskal's gamma:
γ = #Concordant pairs - #Discordant pairs

#Concordant pairs + #Discordant pairs
Pair#teamsCommon    ConcordantDiscordant#pairsγτ%Concordant
The AP poll does track the computers better than the Coaches poll, but the ones that are most alike are the ones that are calculated more nearly the same way.