Rating Games Played - Upsets

© Copyright 2007, Paul Kislanko

6 September, 2007
Well, the first weekend of 2007 certainly did not disappoint. Unless you were a Michigan fan who really thought the Wolverines were a top ten team, I suppose. Many commentators, including some who should know better, immediately labeled Appalachian State 34 Michigan 32 as the biggest upset of all time. Historic, yes - biggest, no.

It was just a few days before the first game when I published a technique to rate upsets on an absolute scale. Weekend one provides some excellent examples on the difference between "a surprise" and an upset. When a lower-ranked team beats a higher-ranked team, it's a surprise. How much of an upset it is depends upon the final rankings - maybe the teams were just ranked incorrectly.

For example, based upon the Pre-Season Bucklin Majority of the computer rankings tracked by Kenneth Massey there were 12 "surprises" in week one.
SF VR HR Date Visitor VS Home HS OT
14.13 121 8 9/1/2007 Appalachian St 34 @ Michigan 32
1.04 100 49 9/1/2007 Bowling Green 32 @ Minnesota 31 1
1.02 53 71 9/1/2007 Virginia 3 @ Wyoming 23
0.74 32 27 9/1/2007 Georgia Tech 33 @ Notre Dame 3
0.36 94 69 9/1/2007 Central Florida 25 @ North Carolina St 23
0.36 69 94 9/1/2007 New Mexico 6 @ UTEP 10
0.34 121 90 9/1/2007 Nicholls St 16 @ Rice 14
0.33 55 58 9/1/2007 Central Michigan 7 @ Kansas 52
0.20 97 88 8/30/2007 Kent St 23 @ Iowa St 14
0.17 92 100 9/1/2007 Middle Tennessee St 14 @ Florida Atlantic 27
0.08 97 90 8/30/2007 Miami OH 14 @ Ball St 13
0.07 30 32 9/3/2007 Florida St 18 @ Clemson 24
Sure enough, the Mountaineers' win at Michigan on an absolute scale would be very slightly bigger than the biggest upset of 2006 (Auburn's win over the eventual champion.)

But what if Michigan wasn't the eighth-best team in the first place? After all the computers take the first week's results into account, there are only four games for which the results are inconsistent with the ratings:
SF VR HR Date Visitor VS Home HS OT
3.48 121 27 9/1/2007 Appalachian St 34 @ Michigan 32
0.22 82 67 9/1/2007 Bowling Green 32 @ Minnesota 31 1
0.19 121 102 9/1/2007 Nicholls St 16 @ Rice 14
0.05 77 81 9/1/2007 New Mexico 6 @ UTEP 10
Appalachian State-Michigan is still the biggest upset so far, but nowhere near the biggest of last season.

Of course, Michigan can move up a lot as the year progresses, or down. How big this will be seen in hindsight won't really be known for another 11 or 12 games.

When a game's results don't match a rating's rankings after the game is played, the game is referred to as a retrodictive ranking violation. We can use the surprise factor to weight the RRVs by team and by rating:

AvgWV Team UCS HAW MAR BAS CLA RUD CGV SAG KEE PFZ LAZ MAS BMC CPA CPR BDF BMaj BIL GM LAW BRN MOR HOW SOL ERD DWI DES D1A PIG DOK DKC SOR NUT KAM CSL
4.7785 Michigan 0.1687 0.7423 1.5847 1.3269 1.4694 1.7500 1.5745 2.1256 2.5588 3.3333 2.6667 3.4815 2.6667 3.8400 3.8400 2.2727 3.4815 3.6538 3.1724 3.3214 3.8400 4.5000 5.7222 5.5000 5.5000 7.6667 4.5000 5.5000 12.4444 1.0000 2.9323 14.1250 14.1250 19.1667 5.7222
0.6184 Kansas                                                     1.7570 2.3478     7.4595       1.8000
0.6184 Central Michigan                                                     1.7570 2.3478     7.4595       1.8000
0.3121 Minnesota             0.1216       0.3913 0.6667 0.2785     0.7292 0.2239   0.3167 0.5769 0.8000 0.9778 0.2453 0.3731 0.8679   0.1324 0.5424 0.1944 1.1220 0.4925 0.7414 1.2444 0.3666 0.2278
0.3121 Bowling Green             0.1216       0.3913 0.6667 0.2785     0.7292 0.2239   0.3167 0.5769 0.8000 0.9778 0.2453 0.3731 0.8679   0.1324 0.5424 0.1944 1.1220 0.4925 0.7414 1.2444 0.3666 0.2278
0.2848 Rice 0.4313 0.3175 0.1524 0.2542 0.1192 0.5217 0.2222 0.4313 0.1416 0.9991 0.1237 0.8357 0.4698 0.2542 0.3419 0.1748 0.1863 0.4578 0.1863 0.3444 0.1524 0.2642 0.2100 0.7423 0.1863 0.2474 0.8597 0.7423 0.1384 0.3297 0.4448 0.4757 0.2872 0.3152 1.5283
0.1183 New Mexico     0.3896       0.6667   0.2933   0.2667         0.4366 0.5195   0.3934 0.6667 0.2000 0.3333   0.2833 0.2468   0.1343 0.8974 0.5263 0.4366   0.7588 0.5128 0.1765 0.9333
0.1183 UTEP     0.3896       0.6667   0.2933   0.2667         0.4366 0.5195   0.3934 0.6667 0.2000 0.3333   0.2833 0.2468   0.1343 0.8974 0.5263 0.4366   0.7588 0.5128 0.1765 0.9333
0.1156 Georgia Tech                                                             2.9565 1.9999    
0.1156 Notre Dame                                                             2.9565 1.9999    
0.0853 UCF                     0.2742   0.1364           0.4219   0.1486 0.1842 0.1184   0.3288   0.6897     0.4424   0.6250 0.5246 0.2754
0.0853 North Carolina St                     0.2742   0.1364           0.4219   0.1486 0.1842 0.1184   0.3288   0.6897     0.4424   0.6250 0.5246 0.2754
0.0764 Miami-Ohio                               0.2273     0.9323 0.4494 0.2779 0.2727   0.1444 0.2400   0.4545 0.3133     0.2198 0.1721 0.2857   0.2750
0.0764 Ball State                               0.2273     0.9323 0.4494 0.2779 0.2727   0.1444 0.2400   0.4545 0.3133     0.2198 0.1721 0.2857   0.2750
0.0610 Florida Atlantic                                             0.2225       0.4545       0.6667 0.3250 0.2245   0.2623
0.0610 Middle Tenn St                                             0.2225       0.4545       0.6667 0.3250 0.2245   0.2623
0.0538 Kent St                     0.2390                   0.6818 0.6522 0.1840   0.6897         0.7297     0.7429  
0.0538 Iowa State                     0.2390                   0.6818 0.6522 0.1840   0.6897         0.7297     0.7429  
0.0518 Arizona St                                                                     1.8140
0.0518 San Jose St                                                                     1.8140
0.0438 Clemson                         0.3769     0.1250             0.1154 0.1667         0.6667 0.7500        
0.0438 Florida St                         0.3769     0.1250             0.1154 0.1667         0.6667 0.7500        
0.0413 Wyoming                                           0.9524   0.1111           1.2391        
0.0413 Virginia                                           0.9524   0.1111           1.2391        
0.0279 Oregon                                                     0.6000               0.3750
AvgWV Team UCS HAW MAR BAS CLA RUD CGV SAG KEE PFZ LAZ MAS BMC CPA CPR BDF BMaj BIL GM LAW BRN MOR HOW SOL ERD DWI DES D1A PIG DOK DKC SOR NUT KAM CSL
0.0279 Houston                                                     0.6000               0.3750
0.0190 Colorado             0.6976                         0.7273           0.2656         0.2677      
0.0190 Colorado St             0.6976                         0.7273           0.2656         0.2677      
0.0087 Iowa                                                                     0.3585
0.0087 Northern Illinois                                                                     0.3585
0.0028 Mississippi                                                   0.9722                
0.0028 Memphis                                                   0.9722                
0.0003 Akron                                                           0.1526        
0.0003 Army                                                           0.1526        
0.0000 (84 teams)                                                                    
  WtdViol 0.0001 0.0013 0.0016 0.0018 0.0021 0.0024 0.0027 0.0028 0.0039 0.0041 0.0046 0.0048 0.0050 0.0051 0.0051 0.0052 0.0052 0.0054 0.0060 0.0066 0.0070 0.0088 0.0089 0.0090 0.0092 0.0101 0.0116 0.0119 0.0169 0.0198 0.0200 0.0224 0.0230 0.0267 0.0272
  Viol Counts 2 2 3 2 2 2 5 2 3 2 6 3 5 2 2 6 4 2 6 6 7 8 7 7 7 4 9 6 5 9 8 8 8 5 10

Note that by some ratings the only violations are "unranked" teams' wins and for others there are more violations than shown in the table above. The tables above were based upon essentially the median rankings, and is shown in the column labeled BMaj.