The Bowl Championship Series: A Case Against Subjectively Aggregated Statistics

November 2, 2005 9:33 AM by Tim Swanson | Other posts by Tim Swanson | Comments (7)

System has fallen and it can't get up:

The BCS means well. All it really wants to do is arrange a No. 1 vs. No. 2 national championship game. But how can you trust a "system" that does more makeovers than the Queer Eye guys? Hardly a year goes by that the BCS doesn't go under the knife for a nip and tuck.

This season, it was the introduction of the 114-member Harris Interactive College Football Poll and the departure of the Associated Press poll. Before that, the lineup of computer rankings was reduced. Before that, the number of components used in the overall rankings system was trimmed. Before that, formulas were adjusted.

It's always something. Strength of schedule is in. Strength of schedule is out. Quality wins are in. Quality wins are out. Congress barks about antitrust legislation. The BCS magically creates new opportunities for lesser conferences.

I'd rather clean Bevo's stall after the big lug scarfs a 10-pack of White Castle Slyders than endure another annual controversy, another cluster of inevitable BCS fixes. The BCS doesn't set the postseason agenda, it reacts to it. Every season a flaw is exposed, followed months later by another bandage applied by the BCS administrators.

A quick note: out of nearly 80 other collegiate sporting varsity championships, Division IA football is the only one in which a winner is not determined through some kind of playoff-bracket system. Instead, heterogeneous teams and conferences of wildly disparate resources and qualities (e.g. Army versus USC, Conference USA versus SEC) are ranked according to a triumvirate of aggregated statistics.

Additionally, instead of utilizing a Sweet 16 or Elite 8 bracket tournament, the BCS uses conference champs plus two-at-large wild cards. There are six conferences which are considered the big kids on the block: Pac 10, SEC, Big 10, Big 12, ACC and Big East. Whoever ends up number one in each of those is given an automatic bid to one of the four BCS bowl games (Sugar, Rose, Orange, Fiesta). So regardless as to how good a team is in relation to the rest of those in Division I-A (as ranked by the Harris Poll or USA Today), as long as they win that conference championship they get a bling bling filled bowl trip (each team that goes to a BCS bowl receives $11-14 million).

[Note: in terms of relative credibility and openness of either the Harris Poll or USA Today poll, both arguably pail in comparison to the new Master Coaches Survey].

Polls Schmolls

This still misses the heart of the problem: that for the same reasons that the US News & World Reports rankings of college academic programs is flawed, so to is the BCS. The scale methodology is flawed if nothing else for the fact that you cannot use a subjectively assessed interval aggregation system to compile a nice round number (see also: The Flaw of Averages).

For instance, the typical Likert-scale uses a 1-5 rating system (1 for worst, 5 for best). Each pollster has their own attitudinal preferences which are subjective and relative to their own internal ranking system. It is foolhardy and misleading to attempt to aggregate all the pollsters subjective views on what flavor of ice cream is the best flavor. Is Crimson Tide Cookies 'n Cream? What about Hokie Orange Sherbert? Can one forget Knute Rockne Fudge Delight?

Similarly for the BCS, the method behind its obfuscated madness is strewn with vain attempts of using a statistical approach to an otherwise scientifically agnostic sport.

Methodology aside, the practicality of implementing a structured playoff system would be both expensive and lengthy -- not to mention making the bowl games less meaningful. Besides, it’s not like the BCS is pitting Toledo and Wyoming in the national title game on a regular basis…

No offense to alums of Toledo and Wyoming.

You may also enjoy reading: Notre Dame Football Announces Improvements To Its Storied History