The first NFL Network game of the year, Jets at Patriots, was a rain-soaked, sordid affair coated with a thick layer of frustration from both teams, highlighted by the Patriots wide receivers' demonstration of how not to catch a football. The Jets, for their part, were lucky the game was close, considering how many times the Patriots settled for field goals or missed them altogether. The final score of 13-10 is a forgiving one. Weeks and years later, the miserable experience of watching this game will have dissipated like the furious breath emanating from Tom Brady's mouth.
That a Thursday Night game on NFL Network lacked the quality we have come to expect from highly-paid athletes at the pinnacle of their sport is nothing new. As Barry Petchesky at Deadspin summarized the day after the game, "…there has seemingly never, ever been a good Thursday game, and there never will be because football players are creatures of habit and don't do well with their between-games routine slashed by three days..." This sentiment is certainly widely shared, but can it be proven? Are Thursday night games actually worse than all other NFL games?
As the Jets-Patriots game demonstrated, final score is not a good enough indicator of game quality. It doesn't tell us enough about what happened before the final whistle. It's just a time slice. Luckily, we live in a world full of smart people, and Brian Burke, founder of Advanced NFL Stats, is one of those smart people. He has two measurements of game quality that can help us, but to understand how they work, we have to take a step back.
Fundamentally, any measure of game quality needs to incorporate what happens during the entire game. This is where Win Probability can help. As explained in the Advanced NFL Stats glossary, Win Probability is: "The probability that a team will win a game in progress, given a particular combination of circumstances including score, time remaining, field position, down and to go distance. WP is based on a model built on actual outcomes of NFL games from recent seasons that featured similar circumstances." What Burke then does is graph these WP changes after each play in the game. Here's the Win Probability graph from the Jets-Patriots Thursday night game:
Win Probability graph for Jets vs. Patriots, 9/12/13 (from Advanced NFL Stats)
You might notice two numbers at the bottom-right: EI and CBF. These are the two primary measures we will use for game quality.
Excitement Index, or EI, is "the total movement of the Win Probability (WP) line during a game. The more that WP fluctuates, the more dramatic, uncertain, and exciting a game is." If a game is decided early (like the Redskins-Packers game from that same week) the line will remain stable for much of the second half and result in a very low EI.
The next measure will be Comeback Factor, or CBF, which is "the inverse of the winning team's lowest Win Probability (WP) during a game. For example, if a winning team's lowest point in a game is 0.10 WP, its CBF would be 10, which is 1 / 0.10. The higher the CBF, the bigger the comeback." For the Jets-Patriots game graphed above, a CBF of 2 is pretty low, because the Patriots' lowest WP was 0.5 (1/ 0.5).
Using data on all games since 2000, I first found the baseline EI and CBF of all NFL games. The average EI is 3.92, and the average CBF is 6.59. Two things to put these numbers in context: EI has a smaller range within the data set (the highest EI -- most exciting game ever? -- was 10.3 when Buffalo defeated Minnesota in 2002 in a 45-39 OT thriller; the lowest ever of 1, shared between four different games, two of which involved the New York Giants), so the average differences within the data set will be relatively small. However, CBF fluctuates much more. The lowest CBF was 1.49, a 2000 game between the Eagles and (again) the Giants. In 13 years of games, there have been 64 CBFs of 100; or, to put another way, there have been 64 instances where the eventual victor only had a 1 percent chance of winning some point in the game, but came back. (Not-coincidentally, 64 games is just about 1 percent of all games played since 2000. Magic, right?)
Back to the question at hand: are Thursday games worse than all other games? All games played on Thursdays have an EI of 3.75, which is 4.3 percent lower than the NFL average. Not a huge difference.
But there are a couple of different ways to look at this. Thursday games also include Thanksgiving, which feature the same two teams every year, one of which is the Lions, who have rarely played entertaining games over the past 13 years. Taking out Thanksgiving games, the EI goes up to 3.99, 1.7 percent greater than the league average.
Thursday night games also include the season openers on NBC, which feature the defending Super Bowl winners against a premier team. Surely those games are more exciting than the average game. Taking out Week 1 Thursday games from the above results, we get an EI of 3.88 -- only 1 percent lower than the league average -- hardly a compelling case for NFL Network games being unanimously worse.
Since it's generally believed NFL Network games are worse because of the creatures-of-habit argument, I compared the numbers to other non-Sunday games. All non-Sunday games have an EI of 3.79, again slightly lower than the league average. Monday Night games, surprisingly enough, have an EI of 3.74, which is almost identical to all Thursday games, but lower than NFL Network games. Considering Monday Night games are often handpicked to be good matchups, this comes as surprising news. Not only are NFL Network games about league average in excitement, they're also above Monday night games.
There is one final consideration to be made with regards to EI. Games late in the season featuring dull matchups may be inherently less exciting, since one or both of the teams have little to play for. Running the same numbers -- but excluding weeks 15, 16 and 17 -- we get a league-wide EI of 3.95, and a Thursday EI of 3.70: Thursdays are 6.3 percent less exciting. But, the Thanksgiving effect is stronger in this data set (because there are fewer data points to drown out the likes of Joey Harrington and Quincy Carter) so Thursday EI rises to 3.97 when Thanksgiving is removed (yes, removing the Lions and Cowboys Thanksgiving games made the NFL 7 percent more exciting). But, taking out the NBC Week 1 games again, and the EI drops back to 3.80. The best argument for boring NFL Network games is that they are, on average, only 4 percent less exciting than an average NFL game.
What NFL Network games have sorely missed are big comebacks. NFL Network games average a Comeback Factor 3.32 -- half the league average -- and only five games with a CBF of 5 or above (where the winning team had a win probability below 20 percent). By definition of the win probability model, 20 percent of the games played should feature a comeback with a CBF of 5 or above (which the larger data set confirms). For NFL Network games, its only 13 percent. For comparison, Monday night games --which often feature hand-picked matchups -- have an average CBF of 8.15, but are right about where they should be in terms of CBF games of 5 or above: 23 percent. The difference is likely that 220 Monday night games have been played since 2000; more games, more chances for unlikely comebacks.
Comebacks are highly correlated with excitement. The real underlying culprit for NFL Network games being so boring, from a statistical perspective, is that the team who gets ahead tends to win. What you see is what you get. The good news is, there have only been 38 non-Thanksgiving NFL Network games so far; it's still a pretty small sample size. Over time, we should see more comebacks, and therefore better games.
But, there are some things EI and CBF can't measure. The Jets-Patriots boondoggle had an EI of 4.4, which we now know to be above league average. Statistically speaking, that game was pretty exciting, and it's easy to see why the stats say so. The Jets had a win probability as low at 8 percent at one point -- just before halftime -- but battled back to a 50 percent WP in the 4th quarter. In a completely objective sense, a 50/50 game midway through the 4th quarter is the basic definition of a good, competitive game.
Of course, this is a textbook case of when stats can't tell us everything. EI or CBF can't account for poor execution, style of play, or the pace of a game, three aspects of a good game Jets-Patriots sorely missed. Looking at the most frequent teams on NFL Network games provides some context: Pittsburgh has nine appearances, Indianapolis has eight, and the following teams have five: 49ers, Jets, Cleveland, New Orleans, Chicago, Denver and Jacksonville. Over the last decade, many of these teams have been below-average or downright terrible. The prevalence of bad teams on Thursday night seems to be the fundamental problem: bad teams usually can't score lots of points in short periods, which prevents big comebacks. This year's lineup still features the Jaguars and Browns, but good matchups, too. It's still early for Thursday Night Football. Stay up late, and the good games will come.