A Better BCS Computer Component?

© Copyright 2008, Paul Kislanko

There are several aspects of the method for handling the BCS computer component that trouble us mathematically-inclined football fans. They have some common root causes, and essentially are there as fixes to the perceived problems rather than real ones.

1 There aren't enough computers

There are 114 Harris voters and 61 voters in the coaches' poll (not all of whom are coaches, of course) for a total of 175 humans. There are only six computers, which makes it problematic to use the same summarization technique that applies to the human polls. There are over a hundred well-understood computer rankings that could be used, but aren't at least partly because...

2 Computers aren't allowed to take MOV into account.

This arbitrary rule eliminates some of the best-known and most widely-respected (for good reason) computer algorithms, including two of the original BCS systems. The rule was added to prevent unsportsmanlike "running up the score" but since there's no similar prohibition (or vetting of any kind) for human voters that make up two thirds of the total formula failing to include these hasn't had any affect at all on the occurrence of RUTS.

One of the reasons an amalgamation of many perspectives tends to give better results in any kind of election than the shared opinions of a few is that "more is better." If we excluded all the human voters who take MOV into account the human polls would be even less useful than they are now.

3 Averaging isn't the right approach

There are really two parts to this.

1. Due to 1 such a small number of values (ranks) gives too much weight to "extreme" outliers
2. Due to 3a there's this silly rule that says "ignore high and low ranks"

I call the rule in 3b "silly" because it recognizes that the averaging method is weak when there are so few values, but doesn't really "fix" the method. A proper fix would add enough computer "voters" to make averaging (or summing Borda counts, which is the same thing) appropriate, or using something more representative than the average. It's also "silly" because no such rule applies to the Harris or Coaches' poll.

Aside:
I wrote a few years ago (before the current formula was adopted) that the computer rankings should be summed up the same way the human polls are. The BCS thinks they are doing that, but they got it wrong. They chose the "25 points for a 1^st place vote down to 1 point for a 25^th place vote" because "that's how the humans do it."

But that's not "how the humans do it." The humans assign (N+1)-R points for each rank R on a ballot that allows N teams. Since the computers rank all 120 teams, "doing it the same way" would mean a vote for 1^st is worth 119, 2^nd 118, down to 0 for 120^th. This is called the "Borda count" and anybody who designs and counts a "poll" should know what they're using and why they've chosen it.

A Better Approach

Just adding five computer rankings and using the same method (drop the high and low and average the rest) gives us:

Rank	Pts	Pct	Team	AND	BIL	COL	JF1†	MB	MLE†	PSR†	SE	WOL	MAS†	SP†
1.	213	0.9467	Texas	4	3	1	2	2	3	1	3	2	1	4
2.	204	0.9067	Alabama	1	4	2	3	3	6	3	2	1	8	6
3.	203	0.9022	Penn State	3	2	4	4	4	4	4	4	3	2	3
4.	198	0.8800	Florida	6	6	3	1	5	1	2	7	6	5	2
4.	198	0.8800	Texas Tech	2	1	5	7	1	8	6	1	4	3	7
6.	184	0.8178	Oklahoma	7	7	6	5	6	5	5	6	5	4	5
7.	168	0.7467	Southern California	11	5	9	6	9	2	8	11	9	7	1
8.	153	0.6800	Boise St	8	10	8	11	10	10	7	8	7	9	11
9.	145	0.6444	Oklahoma St	9	12	12	10	11	7	10	10	12	6	8
10.	140	0.6222	Georgia	10	8	10	8	8	14	12	9	10	13	14
11.	138	0.6133	Utah	5	11	7	13	7	26	9	5	8	15	21
12.	125	0.5556	Ohio State	15	9	13	9	12	11	14	12	11	12	15
13.	121	0.5378	TCU	12	15	11	14	13	12	13	14	13	11	9
14.	113	0.5022	Missouri	13	14	15	17	14	9	15	15	15	10	10
15.	92	0.4089	Ball State	17	25	16	15	20	13	11	13	14	14	20
16.	81	0.3600	Michigan St	14	19	14	16	18	20	16	20	16	16	18
17.	80	0.3556	North Carolina	18	23	18	12	15	16	17	18	18	17	17
18.	43	0.1911	BYU	16	16	17	25	22	46	19	23	17	31	41
19.	39	0.1733	LSU	27	13	29	21	17	34	30	16	22	18	23
20.	38	0.1689	Miami-Florida	20	37	25	18	16	27	24	17	23	19	24
21.	27	0.1200	California	25	22	22	28	25	18	22	34	28	21	16
22.	25	0.1111	Georgia Tech	24	20	21	19	19	36	26	21	26	34	34
23.	24	0.1067	Oregon St	42	18	38	20	28	21	33	28	31	20	19
24.	22	0.0978	Minnesota	22	38	20	34	31	24	18	31	19	23	28
25.	17	0.0756	Pittsburgh	19	36	19	23	34	44	21	33	24	35	40
26.	16	0.0711	Northwestern	21	32	23	27	30	31	23	25	20	22	26
27.	11	0.0489	Iowa	57	54	58	54	59	15	51	59	57	39	13
28.	10	0.0444	Florida St	26	24	27	24	21	30	27	19	25	28	35
29.	9	0.0400	Arizona	53	33	52	47	50	17	49	55	54	36	12
30.	8	0.0356	Tulsa	30	30	26	22	45	22	20	46	32	29	43
31.	6	0.0267	Wake Forest	23	49	28	38	23	42	31	22	29	37	36
32.	4	0.0178	Oregon	44	26	44	50	43	19	38	41	39	27	22
33.	3	0.0133	Cincinnati	28	28	24	37	33	38	25	30	21	33	49
34.	2	0.0089	Virginia	38	41	37	33	24	59	46	24	43	40	42
35.	1	0.0044	West Virginia	29	21	32	35	32	40	28	38	40	25	31
35.	1	0.0044	Nebraska	43	60	45	45	41	23	37	45	48	32	25
37.	0	0.0000	Kansas	40	17	42	46	42	29	35	43	42	30	29
37.	0	0.0000	Mississippi	36	52	46	36	29	28	43	35	44	24	30
37.	0	0.0000	Illinois	50	46	51	55	51	25	52	52	51	38	27

† indicates MOV is considered by the rating.

Just including more and different computer rankings is an improvement, but once we have 11 instead of six computers it's not obvious that we need to drop any of them from the average. It turns out that we get a better result if we don't drop any ranks once we have "enough" ratings. However, we get an even better result if we don't use average rank at all.

Although dropping the high and low ranks before taking the average does nothing except suggest taking the average is a bad idea, if we chose the Bucklin Majority when there's an odd number of ratings we're essentially dropping highs and lows until we select the median. With 11 ratings we're essentially dropping the high five and the low five, but now we're not really "dropping" any, we're selecting the median instead of the mean.

It turns out that picking the median value of six non-MOV and five MOV rankings is a better characterization of the results-to-date than either variety of average ranks:

Rankings in Weighted Retrodictive Ranking Violation Order
6 Nov 2008 11:36:20 (US Central)
Through games of 1 Nov 2008

Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Texas	3	2	2	1	1	4	2	1	3	1	4	1	2	3
Alabama	2	3	3	3	2	1	1	8	4	2	6	3	3	6
Penn State	4	4	4	2	3	3	3	2	2	4	3	4	4	4
Texas Tech	1	1	4	5	4	2	4	3	1	5	7	6	7	8
Florida	7	5	5	4	4	6	6	5	6	3	2	2	1	1
Oklahoma	6	6	5	6	6	7	5	4	7	6	5	5	5	5
Southern California	11	9	8	7	7	11	9	7	5	9	1	8	6	2
Boise St	8	10	9	8	8	8	7	9	10	8	11	7	11	10
Utah	5	7	9	11	11	5	8	15	11	7	21	9	13	26
Georgia	9	8	10	10	10	10	10	13	8	10	14	12	8	14
Oklahoma St	10	11	10	9	9	9	12	6	12	12	8	10	10	7
Ohio State	12	12	12	12	12	15	11	12	9	13	15	14	9	11
TCU	14	13	13	13	13	12	13	11	15	11	9	13	14	12
Missouri	15	14	14	14	14	13	15	10	14	15	10	15	17	9
Ball State	13	20	15	15	15	17	14	14	25	16	20	11	15	13
Michigan St	20	18	16	16	16	14	16	16	19	14	18	16	16	20
North Carolina	18	15	17	17	17	18	18	17	23	18	17	17	12	16
BYU	23	22	22	21	20	16	17	31	16	17	41	19	25	46
California	34	25	22	20	20	25	28	21	22	22	16	22	28	18
LSU	16	17	22	18	19	27	22	18	13	29	23	30	21	34
Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Miami-Florida	17	16	23	18	18	20	23	19	37	25	24	24	18	27
Georgia Tech	21	19	24	22	22	24	26	34	20	21	34	26	19	36
Minnesota	31	31	24	25	24	22	19	23	38	20	28	18	34	24
Northwestern	25	30	25	22	23	21	20	22	32	23	26	23	27	31
Florida St	19	21	26	24	24	26	25	28	24	27	35	27	24	30
Oregon St	28	28	28	26	26	42	31	20	18	38	19	33	20	21
Cincinnati	30	33	30	29	28	28	21	33	28	24	49	25	37	38
Tulsa	46	45	30	28	29	30	32	29	30	26	43	20	22	22
Wake Forest	22	23	31	32	30	23	29	37	49	28	36	31	38	42
West Virginia	38	32	32	30	32	29	40	25	21	32	31	28	35	40
Pittsburgh	33	34	33	27	27	19	24	35	36	19	40	21	23	44
South Carolina	26	27	33	31	31	34	35	26	39	36	33	41	26	32
Mississippi	35	29	36	35	34	36	44	24	52	46	30	43	36	28
Maryland	37	37	37	45	43	31	41	50	34	35	65	45	30	69
Kentucky	36	35	39	36	36	41	34	41	29	40	54	39	29	39
Oregon	41	43	39	33	33	44	39	27	26	44	22	38	50	19
Vanderbilt	39	40	39	41	39	33	45	42	56	43	38	36	39	35
Connecticut	40	36	40	37	38	35	38	43	31	33	48	40	40	43
Kansas	43	42	40	34	35	40	42	30	17	42	29	35	46	29
Virginia	24	24	40	37	37	38	43	40	41	37	42	46	33	59
Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Virginia Tech	32	38	41	39	40	46	37	44	27	41	46	44	32	45
Air Force	44	44	42	42	42	32	36	48	42	31	51	32	41	54
Central Michigan	42	46	42	46	45	37	30	49	43	30	72	29	31	70
Navy	29	39	42	44	44	39	27	45	57	34	53	42	43	52
Nebraska	45	41	43	40	41	43	48	32	60	45	25	37	45	23
Duke	27	26	47	47	47	45	47	47	59	50	32	53	53	50
Western Michigan	48	52	47	52	49	47	33	46	70	39	63	34	42	49
Notre Dame	50	48	48	50	51	48	52	52	40	48	37	50	48	47
Arizona	55	50	49	43	46	53	54	36	33	52	12	49	47	17
South Florida	49	49	49	51	50	49	49	53	35	47	45	48	56	41
Illinois	52	51	51	48	48	50	51	38	46	51	27	52	55	25
Rice	53	58	53	55	54	52	46	54	79	49	69	47	51	58
Iowa	59	59	54	49	52	57	57	39	54	58	13	51	54	15
Arkansas	47	47	55	57	56	51	53	55	61	56	70	64	49	74
Boston College	56	54	55	53	53	54	63	57	47	55	44	55	57	48
East Carolina	57	55	58	58	58	58	60	58	48	57	61	56	62	67
San Jose St	54	60	58	60	60	60	50	65	58	54	81	57	52	82
Wisconsin	62	62	58	54	55	59	58	51	53	60	39	58	58	33
Stanford	60	57	59	56	57	61	59	56	45	59	47	60	60	57
Clemson	58	56	61	59	59	62	68	62	44	61	50	66	63	53
Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Louisville	64	61	63	61	61	56	62	69	63	53	78	62	65	71
Troy	73	79	63	62	62	70	55	61	75	63	64	54	61	51
Rutgers	61	63	64	64	64	71	64	63	76	71	57	67	67	64
Colorado	51	53	65	63	63	55	56	72	65	62	82	69	71	83
Fresno St	65	71	65	67	67	63	61	71	68	65	86	61	44	79
Northern Illinois	76	77	66	66	64	72	67	59	102	66	56	59	59	37
Auburn	67	67	67	65	66	68	74	64	55	74	55	72	77	56
Kansas St	66	68	67	68	69	67	66	66	74	73	59	70	80	66
Buffalo	69	70	68	70	68	69	65	68	94	64	68	63	69	63
UL Lafayette	86	86	70	71	71	77	70	60	85	70	71	65	64	62
New Mexico	72	69	72	73	74	66	71	77	67	72	79	76	76	80
Colorado St	70	66	73	79	77	65	73	87	82	67	95	82	72	101
Hawaii	63	64	73	73	72	73	69	76	73	68	85	73	70	91
Texas A&M	71	65	73	72	73	64	75	73	64	69	76	80	79	81
Houston	78	88	74	75	75	80	72	74	86	78	62	68	66	60
Tennessee	74	74	74	69	70	74	80	67	51	84	52	81	68	55
Purdue	68	72	76	76	76	75	77	79	71	76	67	83	83	76
Akron	77	84	77	81	78	78	76	75	98	77	80	75	78	73
Nevada	80	82	78	78	78	76	79	78	91	75	66	71	93	68
Baylor	81	75	80	77	80	79	93	70	80	86	58	87	86	61
Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Louisiana Tech	82	80	81	84	83	83	81	91	66	79	107	79	75	97
UCLA	75	73	81	80	81	81	83	82	50	80	75	86	87	89
Bowling Green	91	91	82	83	82	84	82	83	105	82	74	74	74	75
Arizona St	84	76	84	82	83	94	98	81	62	92	60	92	89	77
Indiana	83	78	85	86	86	86	88	90	77	85	84	88	94	84
Marshall	85	89	85	88	86	85	78	85	83	81	99	84	81	93
Temple	88	90	85	85	85	93	85	80	101	91	73	77	84	65
Arkansas St	94	94	86	87	88	87	86	86	90	83	90	78	73	78
Mississippi St	89	85	89	89	89	88	94	88	69	97	89	95	90	90
Memphis	93	92	90	91	91	92	89	89	97	90	92	89	88	85
UNLV	87	81	90	90	89	82	90	93	92	88	87	94	95	96
UTEP	95	93	91	92	92	90	91	96	84	87	96	91	85	92
Florida Intl	92	99	92	93	93	99	84	84	96	96	94	90	91	87
Florida Atlantic	98	97	95	94	94	95	87	97	87	89	104	85	82	95
Syracuse	90	87	95	98	97	91	95	98	106	93	97	97	92	103
Southern Miss	101	100	96	97	98	100	96	92	100	99	83	93	96	72
Michigan	99	95	97	95	95	97	102	95	72	100	77	99	97	86
North Carolina St	79	83	98	96	96	98	99	94	78	98	91	104	102	104
New Mexico St	96	101	99	99	99	96	92	105	103	94	115	96	99	108
Wyoming	103	96	101	102	102	89	103	110	95	95	117	101	101	117
Team	SE	MB	Bmaj	Nodrop	Drop	AND	WOL	MAS†	BIL	COL	SP†	PSR†	JF1†	MLE†
Middle Tenn St	100	103	102	101	101	103	97	102	99	106	105	102	104	100
Toledo	102	104	102	102	102	106	100	99	107	103	100	100	107	99
UCF	105	106	102	100	100	102	104	104	88	101	98	98	100	102
Ohio	110	111	104	105	105	108	107	103	104	108	93	103	103	88
Army	104	102	105	107	107	104	105	100	113	104	106	108	110	109
Utah St	97	98	105	106	106	101	101	107	112	102	110	105	108	115
Iowa State	107	108	107	104	104	105	111	101	93	110	88	109	111	94
Tulane	109	107	107	110	109	107	109	111	111	105	103	107	105	110
UL Monroe	111	110	107	108	108	109	106	106	108	107	111	106	98	105
Eastern Michigan	112	112	112	113	111	111	110	113	117	111	112	111	116	111
Miami-Ohio	114	114	112	112	111	112	112	109	110	114	109	112	114	112
Washington	106	105	112	109	110	114	113	112	81	112	101	113	118	107
UAB	113	115	114	115	115	113	108	114	115	115	114	114	113	113
Washington St	108	109	114	111	114	110	114	116	89	113	118	117	117	116
Kent St	118	120	115	113	113	117	117	108	109	117	102	115	115	98
San Diego St	115	113	116	118	118	116	120	117	114	120	116	120	112	118
SMU	119	117	116	117	117	115	119	115	116	116	113	116	106	114
Western Kentucky	120	116	116	116	116	121	118	118	120	109	108	110	109	106
Idaho	116	118	118	119	119	118	116	120	118	118	120	118	119	120
North Texas	117	119	119	120	120	119	115	119	119	119	119	119	120	119
WViol	1883.9	1907.9	2037.0	2126.6	2130.3	2207.3	2224.0	2227.6	2232.4	2350.9	2411.3	2434.3	2465.3	2511.8
559	71	70	77	76	77	72	79	86	78	77	93	83	84	98

While it's clear from these results that adding more computers - even some that include MOV - improves the "fit" of computer rankings to results even when averaging with or without dropping best and worst ranks. Still, the best fit comes from using the Majority Consensus (median) rank.

Now, once we find a more correct summary of the computer ranks, we still have to find a better way to incorporate them than 1/3 of the formula, but that's for another essay.