How Predictive?

October 17, 2016

I created the Iterative Strength of Victory variation of Boyd Nation's original Iterattive Strength Rating to study the behavior of a predictive rating relative to a nearly equivalent retrodictive one. In the ratings systems analysis "predictive" essentially means that home field advantage and margin of victory are used to qualify the wins and losses used to form the rating.

Boyd has subsequently added HFA and game location adjustments to the ISR. The one I calculate for college football is the original that did not.

Predictive systems tend to have a larger number of retrodictive ranking violations because even after team B beats team A the algorithm would favor team A on a neutral-field (or home) rematch.

A rating can be "predictive" without actually making any predictions other than the general expectation is a better-ranked team would be favored. Even that can be "fuzzy" since the teams being compared are not equally-connected within the overall games graph. One can generally make the prediction more precise by using the rating as input to another algorithm – I use the ISOV ratings and SOS values to produce the Normalized Scoring report. A byproduct of that algorithm is an expected game score for any team-pair at any location (road/home or neutral.)

I do not gamble on college sports (or anything else except for the occaisional lottery ticket just to contribute to the fund the State has chosen to receive its lottery's profits) but I've used the predictions with some success in "for fun" prediction games. My algorithm wouldn't be much help to a gambler anyway, since it turns out predicted margins of victory that are nearly always in the same direction as and within a fraction of the games' opening lines so it turns out to be as useful as a quarter.

Actually I took that the predicted MOVs were so close to the opening lines as a reason to not discard the algorithm entirely. It is possible that bookies are the exception to my rule that computers are always better than humans - if "the humans" are bookies their ratings may be better than any computer. (More likely they just use better computer rating systems than I do and it is not really an exception.)

Just for fun I decided to track how well my algorithm predicted the games in week seven. The computer predicted 94 games, and picked the winner 69 times (73.4 per cent.) In 32 of the 69 though, the winner won by less than the predicted margin. What's kind of interesting though is that the more accurate the program is at predicting actual scores the more likely it is that the winner does not "cover" the program's predicted margin.

To analyze the predictions I define two metrics:

Score Is negative for a correct prediction with MOV greater than predicted.

Correct prediction Score = Predicted MOV − Actual MOV

Incorrect prediction Score = Predicted MOV + Actual MOV

LoTrio

LoTrio was a prediction game last sponsored by Dr. Dwiggins (DWI on Dr. Massey's list.) It's scoring was based upon how closely the predicted score for each team matches the actual score.

| predicted visitor score − actual visitor score | + | predicted home score − actual home score |

It is interesting that even when the algorithm misses badly on the predicted margin it is ofen very close on at least one of the team scores. Just for fun:

Visitor	Typ	Home	Pred:	Vscr	Hscr	Actual:	Vscr	Hscr	—	Res	AtP	Score	—	ΔV	ΔH	LoTrio
New Mexico	vs	Air Force		24	41		45	40		Miss	Miss	22.1		21	1	22
Western Michigan	at	Akron		38	29		41	0		Hit	Hit	-32		3	29	32
Mississippi Valley State	at	Alabama State		14	39		24	56		Hit	Hit	-6.9		10	17	27
Texas Southern	at	Alcorn State		27	29		20	23		Hit	Hit	-1.2		7	6	13
Southern California	at	Arizona		29	26		48	14		Hit	Hit	-30.4		19	12	30
Alabama A&M	at	Ark Pine Bluff		37	25		40	7		Hit	Hit	-21.3		3	18	21
Mississippi	at	Arkansas		38	29		30	34		Miss	Miss	13.4		8	5	13
South Alabama	at	Arkansas State		28	25		7	17		Miss	Miss	13.2		21	8	29
Kansas	at	Baylor		19	42		7	49		Hit	Hit	-19		12	7	19
North Carolina A&T	at	Bethune-Cookman		43	16		52	35		Hit	Miss	10.5		9	19	28
Colorado State	at	Boise State		17	38		23	28		Hit	Miss	15.9		6	10	16
Ball State	at	Buffalo		37	17		31	21		Hit	Miss	9.8		6	4	10
Mississippi State	at	BYU		21	32		21	28		Hit	Miss	3.9		0	4	4
North Carolina State	at	Clemson		18	33		17	24		Hit	Miss	8.5		1	9	10
Arizona State	at	Colorado		26	39		16	40		Hit	Hit	-11.5		10	1	11
Florida A&M	at	Delaware State		31	18		41	27		Hit	Hit	-0.4		10	9	19
Furman	at	East Tennessee St		28	20		52	7		Hit	Hit	-36.2		24	13	36
Missouri	at	Florida		19	30		14	40		Hit	Hit	-15.4		5	10	15
Wake Forest	at	Florida State		22	35		6	17		Hit	Miss	2		16	18	34
San Diego State	at	Fresno State		38	22		17	3		Hit	Miss	2		21	19	40
Coastal Carolina	at	Gardner-Webb		39	21		17	7		Hit	Miss	8.1		22	14	37
Vanderbilt	at	Georgia		19	30		17	16		Miss	Miss	11.5		2	14	16
Georgia Southern	at	Georgia Tech		20	29		24	35		Hit	Hit	-1.9		4	6	9
Morgan State	at	Hampton		13	40		12	21		Hit	Miss	17.6		1	19	20
UNLV	at	Hawaii		31	33		41	38		Miss	Miss	5.6		10	5	15
Visitor	Typ	Home	Pred:	Vscr	Hscr	Actual:	Vscr	Hscr	—	Res	AtP	Score	—	ΔV	ΔH	LoTrio
Tulsa	at	Houston		22	45		31	38		Hit	Miss	16		9	7	16
Nicholls State	at	Houston Baptist		41	21		33	30		Hit	Miss	17.1		8	9	17
South Carolina State	at	Howard		51	16		14	9		Hit	Miss	29.6		37	7	44
New Mexico State	at	Idaho		26	37		23	55		Hit	Hit	-20.9		3	18	21
Southern Illinois	at	Illinois State		26	27		28	31		Hit	Hit	-1.7		2	4	7
Nebraska	at	Indiana		30	23		27	22		Hit	Miss	2.3		3	1	3
South Dakota	at	Indiana State		37	34		33	30		Hit	Miss	0.4		4	4	7
Southern	at	Jackson State		51	21		28	24		Hit	Miss	25.9		23	3	26
Austin Peay	at	Jacksonville State		13	65		14	34		Hit	Miss	31.4		1	31	31
Liberty	at	Kennesaw St		28	29		36	21		Miss	Miss	16.3		8	8	16
Northwestern State	at	Lamar		16	36		31	32		Hit	Miss	19.7		15	4	20
Duke	at	Louisville		19	49		14	24		Hit	Miss	20.4		5	25	30
Southern Miss	at	LSU		18	38		10	45		Hit	Hit	-14.9		8	7	15
Albany NY	at	Maine		22	25		16	20		Hit	Hit	-0.7		6	5	11
Florida Atlantic	at	Marshall		29	38		21	27		Hit	Miss	3.6		8	11	19
Minnesota	at	Maryland		24	30		31	10		Miss	Miss	27.2		7	20	27
Louisiana Tech	vs	Massachusetts		39	23		56	28		Hit	Hit	-11.8		17	5	22
Central Arkansas	at	McNeese State		34	26		35	0		Hit	Hit	-27.1		1	26	27
Western Carolina	at	Mercer		23	35		24	38		Hit	Hit	-2.1		1	3	4
North Carolina	at	Miami-Florida		19	40		20	13		Miss	Miss	27.5		1	27	28
Kent State	at	Miami-Ohio		27	23		14	18		Miss	Miss	8.6		13	5	18
Northwestern	at	Michigan State		22	22		54	40		Miss	Miss	14.1		32	18	51
Western Kentucky	at	Middle Tenn State		29	32		44	43		Miss	Miss	4.4		15	11	26
Western Illinois	at	Missouri State		27	26		38	35		Hit	Hit	-1.3		11	9	20
Sacramento State	at	Montana		11	61		7	68		Hit	Hit	-11.4		4	7	11
Visitor	Typ	Home	Pred:	Vscr	Hscr	Actual:	Vscr	Hscr	—	Res	AtP	Score	—	ΔV	ΔH	LoTrio
Tennessee-Martin	at	Murray State		36	24		38	31		Hit	Miss	4.7		2	7	9
James Madison	at	New Hampshire		32	21		42	39		Hit	Miss	7.8		10	18	28
Savannah State	at	North Carolina Central		16	31		3	33		Hit	Hit	-14.9		13	2	15
Southern Utah	at	North Dakota		21	28		23	45		Hit	Hit	-15.5		2	17	19
South Dakota State	at	North Dakota State		25	40		19	17		Miss	Miss	16.8		6	23	29
Idaho State	at	Northern Arizona		18	40		7	52		Hit	Hit	-23.6		11	12	24
Central Michigan	at	Northern Illinois		31	31		34	28		Miss	Miss	6.8		3	3	7
Stanford	at	Notre Dame		29	24		17	10		Hit	Hit	-1.9		12	14	27
Eastern Michigan	at	Ohio		27	34		27	20		Miss	Miss	14.2		0	14	14
Kansas State	at	Oklahoma		33	30		17	38		Miss	Miss	23.4		16	8	23
Utah	at	Oregon State		32	22		19	14		Hit	Miss	5.1		13	8	22
Cal Poly	at	Portland State		41	25		55	35		Hit	Hit	-3.5		14	10	24
Monmouth NJ	at	Presbyterian		32	11		13	17		Miss	Miss	24.4		19	6	24
Iowa	at	Purdue		37	18		49	35		Hit	Miss	4.8		12	17	29
Texas-San Antonio	at	Rice		36	22		14	13		Hit	Miss	13		22	9	31
Villanova	at	Richmond		25	23		0	23		Miss	Miss	25.1		25	0	25
Illinois	at	Rutgers		28	27		24	7		Hit	Hit	-15.4		4	20	24
Abilene Christian	at	Sam Houston		24	58		21	48		Hit	Miss	6.8		3	10	13
VMI	at	Samford		25	35		21	55		Hit	Hit	-23.9		4	20	24
Nevada	at	San Jose State		29	29		10	14		Hit	Hit	-3.4		19	15	34
Connecticut	at	South Florida		19	39		27	42		Hit	Miss	5.3		8	3	11
Southeastern Louisiana	at	Stephen F. Austin		28	36		58	34		Miss	Miss	31.2		30	2	31
Rhode Island	at	Stony Brook		5	54		3	14		Hit	Miss	37.8		2	40	43
Virginia Tech	at	Syracuse		46	18		17	31		Miss	Miss	41.6		29	13	42
Alabama	at	Tennessee		33	25		49	10		Hit	Hit	-31.7		16	15	32
Visitor	Typ	Home	Pred:	Vscr	Hscr	Actual:	Vscr	Hscr	—	Res	AtP	Score	—	ΔV	ΔH	LoTrio
Eastern Kentucky	at	Tennessee State		27	31		28	35		Hit	Hit	-3		1	4	5
Eastern Illinois	at	Tennessee Tech		26	30		30	24		Miss	Miss	9.7		4	6	10
Iowa State	at	Texas		31	39		6	27		Hit	Hit	-13.3		25	12	37
West Virginia	at	Texas Tech		38	32		48	17		Hit	Hit	-25		10	15	25
Chattanooga	at	The Citadel		25	21		14	22		Miss	Miss	11.9		11	1	12
Bowling Green	at	Toledo		17	51		35	42		Hit	Miss	27.4		18	9	27
Georgia State	at	Troy		18	36		21	31		Hit	Miss	8.3		3	5	8
Memphis	at	Tulane		35	23		24	14		Hit	Miss	1.8		11	9	21
Northern Colorado	at	UC Davis		38	32		21	34		Miss	Miss	18.7		17	2	19
Temple	at	UCF		25	32		26	25		Miss	Miss	7.7		1	7	8
Appalachian State	at	UL Lafayette		36	21		24	0		Hit	Hit	-9		12	21	32
Texas State	at	UL Monroe		29	34		34	40		Hit	Hit	-0.9		5	6	10
Florida Intl	at	UNC-Charlotte		26	32		27	26		Miss	Miss	6.8		1	6	7
Pittsburgh	at	Virginia		37	27		45	31		Hit	Hit	-4.7		8	4	12
UCLA	at	Washington State		24	35		21	27		Hit	Miss	4		3	8	11
Montana State	at	Weber State		24	26		27	45		Hit	Hit	-16.3		3	19	22
Delaware	at	William & Mary		30	23		17	24		Miss	Miss	13.9		13	1	14
Ohio State	at	Wisconsin		26	19		30	23		Hit	Hit	-0.4		4	4	8
Northern Iowa	at	Youngstown State		20	28		10	14		Hit	Miss	4.2		10	14	25

Correct prediction	Score = Predicted MOV − Actual MOV
Incorrect prediction	Score = Predicted MOV + Actual MOV