Fixing the BCS - Not
Part 2 - the Computers

© Copyright 2004, Paul Kislanko

See Part One for an explanation of the flaws in the way the BCS uses the human polls, and a suggestion that addresses the biggest problems. In this column, we'll look at the computer portion of the formula.

The problems generally cited for the computer rankings and the way they are used in the BCS are:

• Fans and coaches don't understand how they work. This is because most of them are proprietary, and ones that aren't require a minium understanding of statistics and/or probability theory that most sports fans lack (with the notable exception of baseball fans, who evidently can't get enough of the things).
• They are not as accurate as they could be because of the requirement to not use margin of victory. This is a complaint by the computer ranking authors themselves, several of whom have had to implement less accurate (by their definition) versions of their rankings to continue to participate.
• The criteria for inclusion in the BCS is undocumented. A few of the rankings began as well-known newspaper columns, but several only appeared on the national season because they were included in the BCS. No one knows how these are vetted, and apparently there's no (documented) mechanism for adding or removing a ranking.

In some ways, these are fairly weak complaints. Arguments against the computers and for the polls don't depend upon how well we "understand" the voters. Based upon some of their ballots, we might not want to. But the major difference between the voters in the polls and the computers in that ranking is that there are more of them. The BCS doesn't control them, but a major difference is that the Associated Press and AFCA (USA Today/ESPN) do vet their voting rolls. They ensure that there's fair representation by region, by conference affiliation, etc. That combined with the larger numbers of voters involved limits the worst effects of "mistakes" on individual ballots.

There are also some technical things wrong with the way the computer results are used. One huge advantage of the computer rankings is that no matter what factors they use, they use them consistently from week to week. But because there are only six of them, an extraordinarily high or low value for a team has too great an effect, so for each team the highest and lowest rating is dropped. What this means is that the way they've implemented the ranking, the teams are not compared based upon the same criteria. The best thing about computers - consistency - is eliminated by the BCS formula.

Fixing the Computer Rankings

1. Include more computers. There are 98 categorized on WWW Virtual Library - American College Football - Rankings site. Not all authors would want (or possibly be able) to formally participate, but not all are necessary. Probably 25 or so would be representative of all of them, just as 65 AP voters are representative of thousands of reporters.
2. Summarize the results using the same voting technique I recommendedfor the human polls.

There is no need to proscribe using margin of victory any more than there is disqualifying all human voters who only look at the first digits of game scores and therefore think a 21-9 win is closer than a 51-39 win. The key is to vet the makeup to ensure that it is "balanced" with respect to rankings that use MOV and those that use only winning percentage. Five of each variety as characterized by the RFSC page referenced above would be perfectly adequate, and those characterizations could be used to explain the criteria to the fans and coaches without compromising the propriety of the code.

The second point makes it unnecessary to do anything special about "outlier" values for a team, for the same reason it handles the strategic voter. Team A is never ranked higher than team B unless a majority of the computers agree, so no one computer can "throw off" the average. An example using all 98 of the rankings available is included below.

In the next installment I'll propose a better way to combine the results of the human polls and the computer rankings. No correction to either alone would make the BCS formula more useful for selecting the BCS field.

Computer votes counted by IRV
Through Games of 4 Dec 2004

How to read the table

The number in each cell is the cumulative number of rankings that listed the team that high or higher. Rankings are assigned when the (largest) majority of the rankings list the team higher than every remaining team. This is an application of the selection method known as "Instant Run-Off Voting."

There are 98 computers in this "poll", so a majority is 50.

Source: College Football Ranking Comparison

	Team	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25
1	USC	67	90	98
2	Oklahoma	24	85	95	97	97	98
3	Auburn	7	10	65	75	87	93	96	98
4	Texas			4	40	65	91	94	96	97	97	98
5	California		5	19	40	64	81	92	97	97	97	97	97	98
6	Utah		6	11	33	58	80	87	91	93	93	94	94	96	97	98
7	Boise St					5	12	45	55	69	76	81	83	83	83	87	90	91	91	93	94	95
8	Virginia Tech				1	4	10	21	49	63	72	83	89	91	93	95	95	96	98
9	Louisville			2	7	9	12	26	43	55	63	66	73	75	82	86	90	94	96	96	97	97	98
10	Georgia					2	3	9	17	34	49	61	76	85	88	93	95	96	98
11	LSU						1	3	7	11	28	46	64	75	86	92	93	93	95	96	97	98
12	Miami FL				1	1	4	8	11	23	41	53	60	69	84	91	95	96	96	97	98
13	Arizona St						1	4	6	14	17	24	31	48	64	68	71	72	78	84	86	89	90	90	91
14	Iowa						3	4	15	22	24	32	41	53	58	64	69	75	79	82	85	88	92	94	97
15	Texas A&M						1	1	3	4	10	16	24	32	41	51	60	69	72	77	85	88	91	92	93	95
	Team	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25
16	Virginia									3	5	9	9	12	22	34	42	60	77	85	94	96	96	97	97	98
17	Michigan									1	6	8	15	24	28	38	44	54	59	61	66	67	76	83	89	91
18	Florida St										2	4	9	14	16	23	42	51	70	82	86	90	95	98
19	Tennessee									1	2	6	8	13	16	27	37	45	48	57	60	71	74	78	80	82
20	Oklahoma St														2	4	9	17	27	38	50	70	75	78	80	82
21	Texas Tech										1	2	4	5	7	9	12	17	20	27	34	51	78	83	89	91
22	Wisconsin														1	3	11	16	22	28	40	45	53	59	66	73
23	Oregon St												1	2	3	3	5	12	15	23	31	35	42	54	63	65
24	Purdue										1	1	1	1	3	4	6	7	8	13	18	22	25	38	41	52
25	Ohio St										1	1	1	1	3	3	3	3	5	6	6	11	14	20	34	49
	Florida											2	2	3	3	3	6	7	9	11	16	19	22	33	39	47
	North Carolina									1	1	1	1	1	1	1	2	2	3	4	4	7	11	17	24	29
	UCLA															2	2	3	5	5	7	11	12	12	19	24
	Fresno St												1	1	1	1	1	1	5	6	8	10	13	16	20	22
	Colorado																					1	1	4	10	19
	Team	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25
	Pittsburgh																			3	8	8	8	8	8	11
	Boston College																						1	2	7	9
	Texas-El Paso																						1	4	7	8
	West Virginia																						3	5	6	7
	Toledo																			1	1	1	1	4	5	5
	New Mexico																								1	4
	Bowling Green																			1	2	2	2	2	3	4
	Notre Dame																					1	1	2	3	4
	Alabama																	1	1	1	1	1	1	1	1	3
	Georgia Tech																				1	1	1	1	1	3
	Arkansas																									2
	Clemson																			1	1	1	1	2	2	2
	NC State																							1	2	2
	Penn St																						1	1	1	2
	Stanford																						1	1	1	2
	Maryland																					1	1	1	1	1
	Memphis																								1	1
	Syracuse																									1
	Teams / Spot	3	5	7	8	8	14	12	13	15	16	17	15	17	19	19	19	20	21	23	22	22	22	23	25	29