Mon 29 Sep 2008
Recently I was thinking a bit more about how humans perceive trends from random events. When a sports team does really well in a particular year, how much of that can you attribute to actual skill vs sheer random chance? As with most of life’s important questions, this one can be answered via perl script and graphs.
So what would happen if each game in a sports league was decided purely by luck? I used the NFL for this since their relatively small 16 game schedule means less data to sift through. The process is relatively simple:
- I grab the schedule, save it, and parse it into a simple format so I can read it.
- Compute each game of the season, and keep track of the season tally
- At the end of the season, compare the simulated results against real ones
The data file I used was from the 2008 NFL schedule, downloaded here. The simple script to compute the results is here.
The short answer to my question is: actual skill seems to play a large part of the results, as my random outcomes do not look at all like real NFL results.
Simulated: Real (2005) SEA: 12-4 IND: 14-2 CAR: 11-5 SEA: 13-3 OAK: 11-5 DEN: 13-3 TB: 11-5 JAC: 12-4 BAL: 11-5 CIN: 11-5 NYG: 10-6 PIT: 11-5 IND: 10-6 NYG: 11-5 MIN: 9-7 CHI: 11-5 ARI: 9-7 TB: 11-5 NO: 9-7 CAR: 11-5 CIN: 9-7 NE: 10-6 DET: 9-7 KC: 10-6 NYJ: 9-7 WAS: 10-6 CLE: 9-7 MIA: 9-7 KC: 8-8 SD: 9-7 BUF: 8-8 DAL: 9-7 PIT: 8-8 MIN: 9-7 DEN: 8-8 ATL: 8-8 ATL: 8-8 BAL: 6-10 CHI: 7-9 CLE: 6-10 PHI: 7-9 PHI: 6-10 SF: 7-9 STL: 6-10 TEN: 7-9 BUF: 5-11 HOU: 7-9 DET: 5-11 GB: 6-10 ARI: 5-11 NE: 6-10 NYJ: 4-12 WAS: 6-10 TEN: 4-12 DAL: 5-11 OAK: 4-12 STL: 5-11 GB: 4-12 JAC: 5-11 SF: 4-12 MIA: 5-11 NO: 3-13 SD: 4-12 HOU: 2-14
You can see that the distribution of teams is much different in reality than our expectations. Of course, that is just one simulation compared with just one real season. So what happens if we do this a whole bunch of times?
for f in $(seq 1 10000); do perl nfl.pl ; done > results.txt
So I ran 10,000 years of random NFL results, and graphed out the distribution of wins. Meaning, how many teams in 10,000 years achieved 16/16 seasons (answer: 5) and every other win level. As expected the results are a very normalized curve. Trained math or stats geeks probably knew how to compute that curve without running simulations, but this is my blog after all not theirs and I’m rather enjoying this.
Now compare that to a similar graph from the last 4 years of NFL results. The one on the left is individual years, and the right is aggregate results for all four years. Once again this is how many teams finished a season with 7/16 wins, etc. Underneath that is the expected shape of the curves overlaid from the previous graph.
 |
 |
 |
 |
To me at least, this shows pretty clearly the effect that skill has on the results of NFL football. As expected there is clumping in the middle, but the real teams show a lot more tendency towards good and bad results than a random walk would predict.
The aggregate graph shows it best — over four years we would have expected to see 3 teams finish with 12 victories or more but we actually saw 18 teams with such records. The same holds true on the opposite side — many teams are worse than a random hypothesis would require.
Of course, why stop there? What happens to our random if we decide to make a single team’s results decidedly non-random? I plotted a new 10K year resultset with giving one team a 2.5x advantage in each random encounter.
 |
 |
All that did was shift the results to the left a bit, though interestingly enough we do start the see the emergence of a spike around the 14/16 point. Regardless, simply having a good team wasn’t enough to produce bad teams — which means in the NFL, some of the teams really do suck and they can’t blame simple bad luck for all of their troubles.
In the end, my original hypothesis seems to be completely and utterly false. Good teams do much better than a random performance would predict, and an undefeated season should come once every (approx) 1000 years, not twice in 20 like we’ve seen in the NFL.
