Ratings Trends

October 6, 2016

I ought to be a bit embarrassed about the disclaimer I had posted on the site

The ratings I calculate will not be available until the field is well-enough connected for them to make sense.

It had originally said "until the field is connected" but I had constructed the field so that it became connected after week 3. With only two or three games per team and only a fraction of all games matchups between "like" teams, the ratings didn't "look right" so I didn't publish them. The embarrassing bit is that I was contradicting my own statement that "any computer rating no matter how bad is better than any human ranking no matter how 'expert' the human." Here I allowed one human's judgement - decidedly not an expert's - to invalidate two computer ratings' rankings.

This got me thinking again about whether there's an objective way to determine when there is enough data to begin paying attention to a particular rating's ranking or to rankings in general. For the ratings I calculate I suspect the criterion is the average path length between two vertexes of the connected games graph, but would have to guess what the value for "connected enough" would be. For "rankings in general" I wondered if the correlation of one week's composite to the next might be a viable metric. That may turn out to be useful but if the conjecture is right it isn't yet obvious but it does provide an opportunity to revisit rank correlations.

The measure of correlation I use is the distance between two ordinal rankings:

The distance is the number of discordant pairs - the number of pairs where the teams' relative orders are reversed in the two rankings. When the teams are in the same relative order in both lists the pair is said to be concordant. When the teams have the same rank in either list the pair is ignored.

I like this because it's a simple counting statistic (the distance between two ratings is the total number of position swaps it would take to transform one list into the other) and because it is easy to capture the contribution to the distance by each member of the list. I mostly use rank-correlations to compare different rankings (Computer Ranking Correlation to Majority Conssensus) but the same principle applies to comparing successive instances of the same ranking. My conjecture was that we might be able to tell when rankings in general become useful based upon the week-to-week variation in the Computer Rankings by Bucklin Majority ranks.

I really should have known better. Having a different number of ratings from one week to another is not really a problem because the majority consensus definition is based upon the median, and that is fairly stable with respect to number of ratings. The problem is that very quickly the variation from week to week switches from "ratings getting better" to "all ratings taking into account recent results." The week-to-week variation itself is not very large to begin with (less than 10 per cent of the pairwise comparisons are discordant) and it would take more work than I have done to distinguish between the two effects.

Although my guess turns out to not be all that useful through five weeks, in the process of finding that out I produced a couple of reports that taken together provide good examples of how a team's contribution to the distance measure is more useful than just their rank differences. The chart on the left shows the distance contribution of each team between pre-season and week one, week one → week2, etc. along with a symbol that indicates the direction of rank difference from week4 to week5. The chart on the right shows the actual ranks from pre-season through week five.

Both Texas A&M and Ole Miss have the same ranking after week five as after week four but both contribute three to the distance between those week's consensus rankings. In the Aggies' case, that was because teams ranked better than their 7^th fell below them (Houston 6→8, Louisville 6→10 and Stanford 6→14). In Ole Miss' case, the contribution comes from two teams jumping Ole Miss (Miami 18→11, Western Michigan(!) 25→14) and one higher-ranked team falling below (Florida State 11→19.) To make visible why teams moved the way they did the second chart links to the teams' resumes based upon current Majority Consensus ranks.

Majority Consensus
6 Oct 2016

Temporal Correlation

Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
2	Ohio State	1	0	0	0	0	⇔
3	Michigan	4	1	1	1	0	⇑
3	Clemson	1	0	1	1	0	⇑
3	Alabama	0	0	0	0	0	⇓
7	Texas A&M	5	5	11	0	3	⇔
7	Washington	7	6	7	4	5	⇑
7	Tennessee	3	1	2	0	3	⇑
8	Houston	9	1	1	0	3	⇓
10	Louisville	6	9	7	0	3	⇓
11	Miami-Florida	6	4	17	6	5	⇑
13	Wisconsin	11	4	6	4	2	⇓
13	Stanford	1	0	2	2	4	⇓
14	Western Michigan	9	3	9	15	8	⇑
17	Mississippi	4	2	7	6	3	⇔
17	Nebraska	4	3	11	5	3	⇑
19	Florida State	3	0	10	4	5	⇓
20	West Virginia	5	4	7	6	4	⇑
21	Boise State	6	4	11	4	4	⇑
22	Auburn	7	5	10	15	4	⇑
22	Oklahoma	4	1	6	5	4	⇓
23	Baylor	3	2	7	4	7	⇓
23	Florida	5	5	7	6	2	⇑
23	LSU	8	5	2	11	2	⇑
23	North Carolina	11	6	4	13	9	⇑
23	Virginia Tech	9	10	14	16	2	⇑
Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
24	Arkansas	6	7	7	11	5	⇑
27	Colorado	22	14	9	25	4	⇑
29	UCLA	10	3	8	11	7	⇑
31	Maryland	11	9	9	13	31	⇑
32	Air Force	7	8	6	16	18	⇑
32	South Florida	6	5	10	14	11	⇑
36	Utah	3	6	7	4	12	⇓
36	Troy	5	12	13	20	18	⇑
39	Southern California	15	8	11	10	10	⇑
39	Georgia	5	7	3	21	10	⇓
39	TCU	3	13	9	7	7	⇓
40	Washington State	19	6	7	11	17	⇑
41	Georgia Tech	4	8	11	7	11	⇔
43	Arizona State	7	5	8	16	11	⇓
43	North Carolina State	5	7	4	11	21	⇑
44	Memphis	8	4	8	18	11	⇓
46	Oklahoma State	7	25	7	12	13	⇑
46	Navy	3	6	5	10	12	⇓
46	Appalachian State	3	7	23	15	17	⇑
47	California	10	7	13	8	12	⇑
47	Texas Tech	5	8	7	12	13	⇑
48	Pittsburgh	3	5	7	9	12	⇑
49	Toledo	7	6	6	12	17	⇓
50	Penn State	6	10	7	16	14	⇑
50	BYU	3	6	8	8	12	⇑
Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
51	Wake Forest	6	11	8	23	15	⇓
51	Kansas State	5	6	15	9	17	⇓
52	Georgia Southern	4	8	13	11	12	⇑
53	Mississippi State	22	2	7	12	12	⇑
54	San Diego State	5	6	8	10	23	⇓
54	Iowa	6	6	13	11	23	⇓
55	UCF	13	4	3	22	25	⇑
55	Southern Miss	8	8	12	11	7	⇑
56	Indiana	6	9	6	11	8	⇑
59	Oregon	7	8	11	12	18	⇓
60	Tulsa	15	5	9	12	8	⇑
61	Michigan State	5	4	7	33	18	⇓
62	Minnesota	4	9	4	13	18	⇓
63	Northwestern	12	18	8	8	8	⇑
65	Middle Tenn State	11	7	7	5	10	⇑
68	Texas	13	2	14	12	12	⇓
68	Western Kentucky	4	6	9	17	6	⇓
69	Boston College	10	8	13	7	11	⇑
69	Central Michigan	7	19	11	24	6	⇓
70	Temple	26	7	7	6	11	⇑
71	Cincinnati	7	16	14	12	16	⇓
71	Arizona	5	11	8	8	6	⇓
71	Missouri	9	8	6	11	16	⇓
72	Vanderbilt	8	6	5	6	5	⇓
72	Army	18	11	19	13	5	⇑
Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
73	South Carolina	8	10	10	10	5	⇑
74	Notre Dame	6	6	12	30	11	⇓
75	Akron	7	9	18	4	4	⇑
77	Kentucky	11	8	7	12	4	⇑
77	South Alabama	21	8	12	8	19	⇑
80	Ohio	12	11	5	6	7	⇑
80	Tulane	5	10	4	10	11	⇑
83	Louisiana Tech	5	8	10	8	7	⇑
84	East Carolina	13	10	8	9	10	⇓
85	Utah State	8	6	7	8	6	⇔
85	Wyoming	12	2	6	8	15	⇑
85	Virginia	19	8	7	7	9	⇑
88	Rutgers	6	11	6	6	6	⇓
88	Illinois	8	13	10	8	8	⇑
88	Duke	6	19	6	6	16	⇓
89	Eastern Michigan	3	3	8	16	12	⇑
90	Syracuse	10	6	8	10	15	⇓
91	Old Dominion	10	4	2	11	9	⇑
91	Connecticut	5	3	7	9	9	⇓
95	Ball State	7	4	9	9	13	⇓
96	Texas State	16	2	1	8	10	⇑
98	Purdue	9	10	6	12	9	⇓
99	Oregon State	5	7	5	7	9	⇓
99	Marshall	7	13	30	11	6	⇓
100	Iowa State	8	16	3	9	8	⇓
Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
102	Colorado State	8	5	7	6	9	⇓
102	UL Lafayette	11	4	8	7	6	⇓
102	Hawaii	4	4	7	5	16	⇑
103	New Mexico	9	14	5	9	5	⇑
103	SMU	5	1	8	5	9	⇓
104	UNLV	8	2	1	12	12	⇑
105	Idaho	4	4	9	13	6	⇑
105	Northern Illinois	19	8	6	11	6	⇑
110	Nevada	6	9	8	12	13	⇓
110	Georgia State	12	6	7	8	6	⇓
112	UL Monroe	3	3	8	7	7	⇑
113	Massachusetts	6	2	5	7	7	⇑
113	New Mexico State	2	11	10	0	13	⇑
114	Bowling Green	22	6	14	11	9	⇓
114	Kent State	9	5	8	7	7	⇑
117	Kansas	13	7	7	9	9	⇓
117	North Texas	0	0	3	5	5	⇓
117	Arkansas State	10	7	12	17	5	⇓
118	Fresno State	11	6	4	7	12	⇓
118	Buffalo	20	3	7	14	9	⇓
118	San Jose State	8	6	8	14	10	⇓
119	Texas-San Antonio	3	2	7	10	3	⇔
120	Florida Atlantic	5	3	4	7	11	⇓
120	Florida Intl	11	3	6	1	4	⇑
123	UTEP	10	2	11	4	3	⇔
Rank	Team	0→1	1→2	2→3	3→4	4→5	Trend
124	Miami-Ohio	3	2	7	8	4	⇓
125	Rice	10	4	4	9	2	⇓
126	UNC-Charlotte	0	0	0	1	1	⇓
	ΔWeek	505	412	504	616	587

Ranking History

Team	Pre	1	2	3	4	5
Ohio State	4	2	2	2	2	2
Michigan	11	8	7	6	4	3
Clemson	3	3	5	6	5	3
Alabama	1	1	1	1	2	3
Texas A&M	30	26	21	11	7	7
Washington	23	18	13	15	16	7
Tennessee	11	12	9	11	10	7
Houston	19	7	9	6	6	8
Louisville	33	26	17	7	6	10
Miami-Florida	45	39	37	21	18	11
Wisconsin	25	15	11	16	11	13
Stanford	6	6	6	4	6	13
Western Michigan	55	47	48	37	25	14
Mississippi	8	13	16	22	17	17
Nebraska	39	34	31	22	18	17
Florida State	10	6	5	16	11	19
West Virginia	33	30	32	28	24	20
Boise State	34	28	29	24	22	21
Auburn	30	31	28	39	27	22
Oklahoma	4	9	11	18	19	22
Baylor	13	13	15	15	17	23
Florida	26	28	26	27	27	23
LSU	9	16	18	19	30	23
North Carolina	21	29	22	27	34	23
Virginia Tech	44	46	54	40	25	23
Team	Pre	1	2	3	4	5
Arkansas	20	25	18	14	25	24
Colorado	84	62	53	53	31	27
UCLA	27	35	36	30	34	29
Maryland	80	78	69	69	60	31
Air Force	67	69	63	61	49	32
South Florida	50	45	37	29	40	32
Utah	25	25	28	27	24	36
Troy	98	96	86	70	51	36
Southern California	20	33	25	34	41	39
Georgia	21	14	20	20	36	39
TCU	14	14	28	30	30	39
Washington State	37	54	59	51	52	40
Georgia Tech	57	54	52	41	41	41
Arizona State	47	45	37	42	30	43
North Carolina State	53	49	59	56	60	43
Memphis	46	49	49	47	31	44
Oklahoma State	23	19	42	36	48	46
Navy	35	34	38	41	35	46
Appalachian State	55	54	50	70	55	46
California	43	49	58	43	51	47
Texas Tech	51	48	58	52	49	47
Pittsburgh	41	39	39	47	52	48
Toledo	39	30	36	33	33	49
Penn State	48	45	56	52	62	50
BYU	40	37	41	47	51	50
Team	Pre	1	2	3	4	5
Wake Forest	88	85	71	66	43	51
Kansas State	56	57	59	41	39	51
Georgia Southern	54	53	46	58	59	52
Mississippi State	23	47	44	49	58	53
San Diego State	44	39	35	30	33	54
Iowa	22	24	19	32	33	54
UCF	122	111	109	106	85	55
Southern Miss	70	59	62	73	61	55
Indiana	66	59	63	59	64	56
Oregon	22	23	24	32	42	59
Tulsa	88	70	74	67	61	60
Michigan State	12	17	17	14	45	61
Minnesota	60	60	59	56	45	62
Northwestern	43	52	70	64	70	63
Middle Tenn State	81	75	85	79	73	65
Texas	48	34	33	46	53	68
Western Kentucky	37	37	40	49	65	68
Boston College	73	79	71	85	78	69
Central Michigan	75	70	53	41	65	69
Temple	51	79	72	71	79	70
Cincinnati	65	64	51	60	53	71
Arizona	55	59	68	67	67	71
Missouri	61	68	62	62	54	71
Vanderbilt	77	83	74	79	70	72
Army	109	89	79	59	73	72
Team	Pre	1	2	3	4	5
South Carolina	68	63	72	65	75	73
Notre Dame	14	24	22	34	64	74
Akron	90	89	96	79	81	75
Kentucky	78	87	94	94	83	77
South Alabama	110	88	92	102	99	77
Ohio	85	99	89	87	86	80
Tulane	115	114	102	99	91	80
Louisiana Tech	70	68	66	75	85	83
East Carolina	81	70	60	66	77	84
Utah State	77	71	83	78	85	85
Wyoming	113	100	102	96	103	85
Virginia	72	89	89	95	93	85
Rutgers	89	93	87	84	86	88
Illinois	73	66	80	89	93	88
Duke	57	53	72	79	71	88
Eastern Michigan	125	121	124	115	101	89
Syracuse	80	73	84	88	77	90
Old Dominion	112	109	110	112	105	91
Connecticut	85	86	84	81	89	91
Ball State	105	98	99	91	83	95
Texas State	122	105	105	106	107	96
Purdue	93	88	95	97	87	98
Oregon State	96	95	94	89	88	99
Marshall	64	64	57	87	94	99
Iowa State	80	86	100	104	98	100
Team	Pre	1	2	3	4	5
Colorado State	90	101	95	91	91	102
UL Lafayette	107	114	109	101	100	102
Hawaii	121	123	120	119	116	102
New Mexico	94	87	97	103	109	103
SMU	106	102	100	93	96	103
UNLV	108	104	104	104	114	104
Idaho	115	115	116	120	110	105
Northern Illinois	72	89	95	96	109	105
Nevada	88	90	92	86	95	110
Georgia State	100	112	117	108	106	110
UL Monroe	124	122	121	114	113	112
Massachusetts	113	114	117	113	114	113
New Mexico State	122	125	116	123	125	113
Bowling Green	51	71	76	93	106	114
Kent State	111	115	121	116	115	114
Kansas	118	109	117	119	113	117
North Texas	126	126	124	122	116	117
Arkansas State	73	83	91	101	116	117
Fresno State	108	113	107	109	108	118
Buffalo	101	117	118	123	110	118
San Jose State	89	100	94	97	112	118
Texas-San Antonio	117	117	119	112	119	119
Florida Atlantic	106	102	106	111	111	120
Florida Intl	107	114	119	123	127	120
UTEP	114	106	108	119	123	123
Team	Pre	1	2	3	4	5
Miami-Ohio	116	117	120	115	117	124
Rice	106	113	115	114	124	125
UNC-Charlotte	127	128	126	128	125	126