Pairwisee Comparisons

© Copyright 2009, Paul Kislanko

A few years ago an SEBaseball.com subscriber suggested that I create a report for D1 Baseball similar to the one the NCAA uses to select and seed the Hockey tournament. The idea is pretty simple:

  1. identify a subset of the field that is "of interest" (referred to as "teams under consideration")
  2. compare each such team to each of the others based upon a standard set of criteria
  3. compute a "pairwise winning percentage" from the number of teams for which each has a pairwise win.

Although such a report is most useful in sports with a requirement to select and seed a tournament, nothing prevents us from creating one for FBS football. I wouldn't use it by itself to characterize teams' resumes, but for any two teams the reports provide relevant data that is otherwise hard (or at least inconvenient) to obtain.

Defining the field

The obvious choice for "teams of interest" is "teams with .500 or better records." At the end of the season, these are essentially the bowl-eligible teams, but more important for comparison purposes is that a win over any such team is one over a team that has a winning record or would have were it not for said win over the team.

After week 7, there are 81 such teams. In other sports we'd add a second criterion: "ranked in the top N or leading their conference" because in other sports there's an NCAA-specified computer ranking and conference champions get automatic bids regardless of record.

For FBS football there's no tournament with automatic bids, so we don't need the "leading their conference" criterion, and there's no pre-defined computer rating equivalent to baseball (/hockey/basketball) RPI. However, there's a good reason to include the "rank ≤ N" criterion: if weak teams are included in the field, the "wins vs teams under consideration" criterion we'd like to use would give too much weight to teams with wins over teams that are ≥ .500 only because they've played really bad teams other than this one.

So I add to the criteria list for field definition: "ranked in the top half of FBS." Which begs the question "ranked how?" In the absence of an official rating system (not even the BCS considers any version of the RPI for football, for good reasons) by default I use the Majority Consensus rank for the Computer Ranking Comparison maintained by Kenneth Massey.

Comparison Criteria

The "pairwise score" is formed by comparing every team in the field to every other team in the field according to the following criteria:
Head-to-Head
Each team gets 1 point for each win over the other
Record vs common opponents
A team with a better winning percentage against common opponents gets 1 point; if the winning percentage vs common opponents is the same, each team gets ½ point. If there are no common opponents, neither scores.
Record vs teams under consideration
Compare winning percentages using #wins over non-losing teams ÷ (#wins over non-losing teams + all losses). The team with the better record gets 1/4 point.
# Wins vs teams under consideration
The team with more total wins against this field gets 1/2 point.
Record in last 4 games
A team with a better record over the last 4 games than the other gets an eighth of a point. (This serves mainly as a tiebreaker.)
Best overall win
Defeated opponents' that are not common opponents ranks are converted into sextiles (1:20→0, 21:40→1, ... 101:120→5, FCS→6) and if one team's best is lower than the other's that team gets 1/4 point.
Worst overall Loss
Opponents that are not common opponents who defeated the team's ranks are converted into sextiles as above and if one team's worst is lower than the other's that team gets 1/4 point.

Pairwise Winning Percentage

For each of the N FBS teams with non-losing records, count a "pairwise win" over one of the N-1 other teams if its score compared to the other team is higher than the other team's score compared to it. For equal scores, count a "pairwise tie" and otherwise a "pairwise loss."

If we let WW = #pairwise wins, LL = #pairwise losses, and TT = #pairwise ties, then we can define a "pairwise winning percentage" in the usual way:

PW% = (WW+TT/2) ÷ (WW+LL+TT)

Pairwise winning percentage is not likely to distinguish all teams in the "field." As a tiebreaker we can use the difference in total pairwise points (P) between each team ti and every other team tj. If we call the pairwise score of team x vs team y P(tx,ty), we can sum scores over all pairs by:

Delta(ti) = N
∑{ P(ti,tj) − P(tj,ti) }
 j = 1*
*j ≠ i

Of course, this is only one of many possible results analyses, but it is an especially useful one since it makes visible the comparisons that lead to the ranking. For each team, the comparison to every other team under consideration is provided.

For the 2009 report, see Pairwise Ratings - FBS.