Kenneth Massey's College Football Ranking Comparison rates the rankings by how well they correlate to the average of all ranks (by left-to-right order) and includes data for ranking violations. "Rankings violations" are based upon the number of games in which a team currently ranked worse won the game.
For teams, Dr. Massey's page only shows the mean (the top-to-bottom order), median, and standard deviation of the computers' rankings for each team. It occurred to me that I could use the same kind of analysis as for the rankings to find which teams have caused the most trouble for the computers trying to rank them.
Taking the computer rankings*1 and the schedule of games played, for each team I calculated:
Each of these measures a different aspect of how hard a team is to be characterized by a ranking of all teams. Do not mistake these for rankings of team quality. For example, one of the teams for which all of the computers have the team ranked below all of the teams it's lost to and ahead of all the teams it's beaten is Temple, which is pretty much ranked below any team it could possibly play and dutifully lost to all teams it has played.
Through November 13, with 85 computer rankings the top 25 teams in terms of hardest for the computers to deal with are shown below. The "Surprise Factor" is a combination of the number of unexplainable results combined with the magnitude of the "upsets." TCU and UCLA, for instance, make the top 25 because of just one inexplicable loss.
| Team | Rec | Conf | SF | |
| 1 | San Diego St | 4-6 | MW | 17799 |
| 2 | SMU | 3-6 | CUSA | 15307 |
| 3 | Iowa State | 7-3 | B12 | 14479 |
| 4 | Arizona | 3-7 | P10 | 12975 |
| 5 | Florida St | 7-3 | ACC | 12925 |
| 6 | Memphis | 4-5 | CUSA | 12703 |
| 7 | Central Michigan | 5-5 | MAC | 12311 |
| 8 | North Carolina St | 4-5 | ACC | 10871 |
| 9 | Middle Tenn St | 3-5 | SBC | 9161 |
| 10 | Oregon St | 4-6 | P10 | 9057 |
| 11 | Northern Illinois | 5-4 | MAC | 8739 |
| 12 | Pittsburgh | 5-5 | BigE | 8725 |
| 13 | TCU | 10-1 | MW | 8274 |
| 14 | Vanderbilt | 4-6 | SEC | 8048 |
| 15 | Wake Forest | 4-7 | ACC | 7896 |
| 16 | UCLA | 9-1 | P10 | 6474 |
| 17 | Georgia Tech | 6-3 | ACC | 6442 |
| 18 | Ball State | 4-6 | MAC | 6248 |
| 19 | Houston | 5-4 | CUSA | 6245 |
| 20 | Akron | 4-5 | MAC | 6172 |
| 21 | Clemson | 6-4 | ACC | 5260 |
| 22 | South Florida | 5-3 | BigE | 5124 |
| 23 | Oklahoma St | 4-5 | B12 | 5056 |
| 24 | Kansas | 5-5 | B12 | 4986 |
| 25 | Missouri | 6-4 | B12 | 4907 |
Of course, since a team can be on this list because it won a game it shouldn't have, another team might be on it for having lost the same game. The games that have caused the most trouble are shown below. The first ten are off the scale, because no computer has the winner ranked higher even after the fact.
|
| Note 1 | I actually included 7 computer rankings that are not reported by Massey, but the only one worth mentioning is the Sagarin "Predictor" ranking. The others are part of a separate study, and of those only Boyd Nation's ISR really contributes to the conclusions of this article. |
|---|---|
| Note 2 | This metric was suggested by Adam Holtz, who feared I might actually use it for more than he suggested, which I did. |