How Predictive?

October 17, 2016

I created the Iterative Strength of Victory variation of Boyd Nation's original Iterattive Strength Rating to study the behavior of a predictive rating relative to a nearly equivalent retrodictive one. In the ratings systems analysis "predictive" essentially means that home field advantage and margin of victory are used to qualify the wins and losses used to form the rating.
Boyd has subsequently added HFA and game location adjustments to the ISR. The one I calculate for college football is the original that did not.
Predictive systems tend to have a larger number of retrodictive ranking violations because even after team B beats team A the algorithm would favor team A on a neutral-field (or home) rematch.

A rating can be "predictive" without actually making any predictions other than the general expectation is a better-ranked team would be favored. Even that can be "fuzzy" since the teams being compared are not equally-connected within the overall games graph. One can generally make the prediction more precise by using the rating as input to another algorithm – I use the ISOV ratings and SOS values to produce the Normalized Scoring report. A byproduct of that algorithm is an expected game score for any team-pair at any location (road/home or neutral.)

I do not gamble on college sports (or anything else except for the occaisional lottery ticket just to contribute to the fund the State has chosen to receive its lottery's profits) but I've used the predictions with some success in "for fun" prediction games. My algorithm wouldn't be much help to a gambler anyway, since it turns out predicted margins of victory that are nearly always in the same direction as and within a fraction of the games' opening lines so it turns out to be as useful as a quarter.

Actually I took that the predicted MOVs were so close to the opening lines as a reason to not discard the algorithm entirely. It is possible that bookies are the exception to my rule that computers are always better than humans - if "the humans" are bookies their ratings may be better than any computer. (More likely they just use better computer rating systems than I do and it is not really an exception.)

Just for fun I decided to track how well my algorithm predicted the games in week seven. The computer predicted 94 games, and picked the winner 69 times (73.4 per cent.) In 32 of the 69 though, the winner won by less than the predicted margin. What's kind of interesting though is that the more accurate the program is at predicting actual scores the more likely it is that the winner does not "cover" the program's predicted margin.

To analyze the predictions I define two metrics:

Score Is negative for a correct prediction with MOV greater than predicted.
Correct predictionScore = Predicted MOV − Actual MOV
Incorrect predictionScore = Predicted MOV + Actual MOV
LoTrio
LoTrio was a prediction game last sponsored by Dr. Dwiggins (DWI on Dr. Massey's list.) It's scoring was based upon how closely the predicted score for each team matches the actual score.
| predicted visitor score − actual visitor score | + | predicted home score − actual home score |
It is interesting that even when the algorithm misses badly on the predicted margin it is ofen very close on at least one of the team scores. Just for fun:
VisitorTypHomePred:VscrHscrActual:VscrHscrResAtPScoreΔVΔHLoTrio
New MexicovsAir Force24414540MissMiss22.121122
Western MichiganatAkron3829410HitHit-3232932
Mississippi Valley StateatAlabama State14392456HitHit-6.9101727
Texas SouthernatAlcorn State27292023HitHit-1.27613
Southern CaliforniaatArizona29264814HitHit-30.4191230
Alabama A&MatArk Pine Bluff3725407HitHit-21.331821
MississippiatArkansas38293034MissMiss13.48513
South AlabamaatArkansas State2825717MissMiss13.221829
KansasatBaylor1942749HitHit-1912719
North Carolina A&TatBethune-Cookman43165235HitMiss10.591928
Colorado StateatBoise State17382328HitMiss15.961016
Ball StateatBuffalo37173121HitMiss9.86410
Mississippi StateatBYU21322128HitMiss3.9044
North Carolina StateatClemson18331724HitMiss8.51910
Arizona StateatColorado26391640HitHit-11.510111
Florida A&MatDelaware State31184127HitHit-0.410919
FurmanatEast Tennessee St2820527HitHit-36.2241336
MissouriatFlorida19301440HitHit-15.451015
Wake ForestatFlorida State2235617HitMiss2161834
San Diego StateatFresno State3822173HitMiss2211940
Coastal CarolinaatGardner-Webb3921177HitMiss8.1221437
VanderbiltatGeorgia19301716MissMiss11.521416
Georgia SouthernatGeorgia Tech20292435HitHit-1.9469
Morgan StateatHampton13401221HitMiss17.611920
UNLVatHawaii31334138MissMiss5.610515
VisitorTypHomePred:VscrHscrActual:VscrHscrResAtPScoreΔVΔHLoTrio
TulsaatHouston22453138HitMiss169716
Nicholls StateatHouston Baptist41213330HitMiss17.18917
South Carolina StateatHoward5116149HitMiss29.637744
New Mexico StateatIdaho26372355HitHit-20.931821
Southern IllinoisatIllinois State26272831HitHit-1.7247
NebraskaatIndiana30232722HitMiss2.3313
South DakotaatIndiana State37343330HitMiss0.4447
SouthernatJackson State51212824HitMiss25.923326
Austin PeayatJacksonville State13651434HitMiss31.413131
LibertyatKennesaw St28293621MissMiss16.38816
Northwestern StateatLamar16363132HitMiss19.715420
DukeatLouisville19491424HitMiss20.452530
Southern MissatLSU18381045HitHit-14.98715
Albany NYatMaine22251620HitHit-0.76511
Florida AtlanticatMarshall29382127HitMiss3.681119
MinnesotaatMaryland24303110MissMiss27.272027
Louisiana TechvsMassachusetts39235628HitHit-11.817522
Central ArkansasatMcNeese State3426350HitHit-27.112627
Western CarolinaatMercer23352438HitHit-2.1134
North CarolinaatMiami-Florida19402013MissMiss27.512728
Kent StateatMiami-Ohio27231418MissMiss8.613518
NorthwesternatMichigan State22225440MissMiss14.1321851
Western KentuckyatMiddle Tenn State29324443MissMiss4.4151126
Western IllinoisatMissouri State27263835HitHit-1.311920
Sacramento StateatMontana1161768HitHit-11.44711
VisitorTypHomePred:VscrHscrActual:VscrHscrResAtPScoreΔVΔHLoTrio
Tennessee-MartinatMurray State36243831HitMiss4.7279
James MadisonatNew Hampshire32214239HitMiss7.8101828
Savannah StateatNorth Carolina Central1631333HitHit-14.913215
Southern UtahatNorth Dakota21282345HitHit-15.521719
South Dakota StateatNorth Dakota State25401917MissMiss16.862329
Idaho StateatNorthern Arizona1840752HitHit-23.6111224
Central MichiganatNorthern Illinois31313428MissMiss6.8337
StanfordatNotre Dame29241710HitHit-1.9121427
Eastern MichiganatOhio27342720MissMiss14.201414
Kansas StateatOklahoma33301738MissMiss23.416823
UtahatOregon State32221914HitMiss5.113822
Cal PolyatPortland State41255535HitHit-3.5141024
Monmouth NJatPresbyterian32111317MissMiss24.419624
IowaatPurdue37184935HitMiss4.8121729
Texas-San AntonioatRice36221413HitMiss1322931
VillanovaatRichmond2523023MissMiss25.125025
IllinoisatRutgers2827247HitHit-15.442024
Abilene ChristianatSam Houston24582148HitMiss6.831013
VMIatSamford25352155HitHit-23.942024
NevadaatSan Jose State29291014HitHit-3.4191534
ConnecticutatSouth Florida19392742HitMiss5.38311
Southeastern LouisianaatStephen F. Austin28365834MissMiss31.230231
Rhode IslandatStony Brook554314HitMiss37.824043
Virginia TechatSyracuse46181731MissMiss41.6291342
AlabamaatTennessee33254910HitHit-31.7161532
VisitorTypHomePred:VscrHscrActual:VscrHscrResAtPScoreΔVΔHLoTrio
Eastern KentuckyatTennessee State27312835HitHit-3145
Eastern IllinoisatTennessee Tech26303024MissMiss9.74610
Iowa StateatTexas3139627HitHit-13.3251237
West VirginiaatTexas Tech38324817HitHit-25101525
ChattanoogaatThe Citadel25211422MissMiss11.911112
Bowling GreenatToledo17513542HitMiss27.418927
Georgia StateatTroy18362131HitMiss8.3358
MemphisatTulane35232414HitMiss1.811921
Northern ColoradoatUC Davis38322134MissMiss18.717219
TempleatUCF25322625MissMiss7.7178
Appalachian StateatUL Lafayette3621240HitHit-9122132
Texas StateatUL Monroe29343440HitHit-0.95610
Florida IntlatUNC-Charlotte26322726MissMiss6.8167
PittsburghatVirginia37274531HitHit-4.78412
UCLAatWashington State24352127HitMiss43811
Montana StateatWeber State24262745HitHit-16.331922
DelawareatWilliam & Mary30231724MissMiss13.913114
Ohio StateatWisconsin26193023HitHit-0.4448
Northern IowaatYoungstown State20281014HitMiss4.2101425

© Copyright 2016, Paul Kislanko