On Sunday during the most watched show in US television history, 114 million viewers were treated to an extraordinarily entertaining Super Bowl. The Patriots and Seahawks provided fans with one of the most dramatic finishes in NFL history. Armchair quarterbacks everywhere are still questioning Pete Carroll’s ill-fated decision to pass the ball on the one yard line rather than simply handing it to Marshawn Lynch. One NFL veteran went as far as calling it “the dumbest play call in the history of NFL football.” Decisions that are judged solely upon their outcomes are called “playing results”, and this is exactly what appears to be occurring in the court of public opinion at the moment.
For over a decade there has been a groundswell of evidence (much of it documented by members of our Edj Analytics group) stating that NFL coaches are too conservative in their play-calling. In particular, 4th down decisions made on the field often disagree with technical analysis. Although Sunday’s infamous decision didn’t occur on 4th down, it does share much in common with those plays.
It has been our observation while analyzing play-calling decisions in the NFL for the past decade that optimum choices often get muddled by conventional wisdom. Here are some ways this occurs:
- Coaches commonly rely upon experience, feel, and the lessons handed down from the prior greats of the game. Banking on the human brain to sort through a vast array of real-time on-field variables and historical results is a monumental task. When it comes to processing large sets of data to optimize decisions in competitive environments, machines win over humans every time. It’s not that human experience and intuition don’t matter, it is simply a function of processing speed and accuracy. Just ask the world chess champion Magnus Carlsen who is unable to compete at the level of the best neural net program. Despite all of the evidence to the contrary, we still hear the standard retort from NFL coaches past and present: “football is played by men, not computers.”
- Although everyone agrees that the only goal in any football game is to finish with a W, there is a very strong external force that argues to the contrary: the opinion of others. This group includes the players, the owner, the media, and the fans. A play call that goes against common accepted practice, even if it is the most likely to win the game, will be second-guessed far more harshly when the result leads to a loss compared to the play accepted as correct by conventional wisdom. Psychologists refer to this as the risk-aversion bias.
- When faced with a decision which will win or lose immediately compared to one that prolongs the game, the tendency among the vast majority of games players (in particular football coaches) is to choose the second, even when the first has the higher expectation of winning. Again, due to risk averse tendencies, a coach can more easily distance himself from a failure if it occurs more gradually.
Add to these one subtle discovery of psychologists and behavioral economists (ref. Predictably Irrational, The Hidden Forces That Shape Our Decisions by Daniel Ariely): humans count a reward as in their possession before it actually is. The spectators already gave Seattle the trophy when they made it to the 1-yard line with 2nd and goal, 26 seconds remaining. This is similar to coaching decisions made based upon the fallacy that a field goal represents “guaranteed points”. In fact, according to our simulation model, the Seahawks were only about a 3:1 favorite to win prior to their final offensive play.
There are two components to the analysis of any game decision, the technical part and the opponent adjustment part. Let’s look at both in that order. 10+ years ago we built a tool (Zeus) that determines expected outcomes of NFL play call decisions. It is based upon two things – the historical performance of NFL teams under game conditions and the customization of the teams involved, reflecting their year-to-date performances adjusted for strength of opponent. Zeus is not a lookup table, far from it. Distributions of single play outcomes have been developed based upon (but not limited to) historical performances under varying conditions including ball location, time on the clock, score, down and distance. Then hundreds of thousands of simulated games are executed to completion using the expected outcomes of those individual plays adjusted for the two teams’ performance levels. Seattle’s play call decision is more than just the 1-dimensional view that “running the ball is right because Seattle has the best offensive rushing team in the NFL.” New England’s rushing defense, Seattle’s passing offense, and New England’s passing defense also enter the hopper.
The result of a 1,000,000 trial Zeus simulation (customized to the full season performances of Seattle and New England) is statistically conclusive: passing wins 0.8% more games with an (2 standard deviation) uncertainty of 0.12%. Yes, that is with Beast Mode present. The simulation model is accounting for the many variations that occur with a single time out and only 26 seconds on the clock. The technical side of the decision has been reached.
The second component of the decision has an academic name: game theory. Its formal study and practice were developed for a graver situation – the deployment of nuclear weapons (ref. Prisoner’s Dilemma by William Poundstone). Economist Justin Wolfers has already expounded upon this component of Sunday’s game in a recent New York Times article.
Football decision-making has been compared to similar actions in chess, but poker may be more accurate. In a game with hidden information, the best play is often the one the opponent isn’t expecting, whether or not that is the correct technical play. There are NFL defensive coordinators who have cemented careers by studying the predictable offensive play call tendencies of opponents. What do the majority of NFL coaches call at the opponent’s 1-yard line? What is New England expecting? What should Seattle do?
In recent NFL history when teams faced 2nd and 1 from the opponent’s 1, nearly 70% chose to run. Likely many of the 30% who passed were influenced to do so because of a weak running attack. Would New England’s defensive setup be influenced by this statistic?
As everyone now knows, Seattle chose the poker pro’s play, the one that few spectators expected, the call that would lead to a roar never heard in NFL history if it went wrong. But the play call is only the beginning. The opposing team has a decision to make on their setup. Do they bias for the run or the pass? The offense still must correctly execute and the defense must react. How much of the end result was due to the actions and reactions of the 22 players on the field as opposed to the coaches? The answers to those questions are left to others.
The “worst play call in NFL history” may have actually been the best, correctly seen independently of the result. Two of the greatest coaches in NFL history stared each other down in a heads-up poker game with massive consequences and neither blinked. Hopefully after the smoke of criticism clears, this contest will come into clear view as the most memorable occurrence of the greatest game in Super Bowl history.