GREEN BAY – On the final play of the first half last Sunday against the Carolina Panthers, Green Bay Packers coach Matt LaFleur went for the touchdown from the 1-yard line rather than kick the gimme field goal.
In the fourth quarter, Carolina scored a touchdown to cut a 14-point lead to eight. Panthers coach Ron Rivera went for the two-point conversion rather than kicking the extra point.
It too failed.
After the game, LaFleur said his decision was from the gut. Rivera said his was all analytics. As it turns out, despite plenty of criticism, both made the right call.
If you keep an eye on Twitter during the NFL season, you might be at least vaguely familiar with the term win probability. It’s a tool NFL analytics experts — and many teams — use to make or evaluate game-management decisions. Should you punt or go for it on fourth down? Or kick a field goal? Should you kick the extra point or go for two? Decisions like that.
Win probability can help by crunching all sorts of data into an algorithm. Factors such as the score, time left in the game, who gets the ball to start the second half, the quality of the offenses and defenses for both teams, home-field advantage and the like. It’s based on years of NFL play-by-play and statistical data and can provide a win probability for any circumstance in an NFL game.
For example, if the algorithm says that attempting a field goal on fourth down from a given yard line at a given time gives you a 60 percent chance of winning, and going for it gives you a 65 percent chance of winning, the better decision is to go for it. Remember, those probabilities account for all outcomes (i.e., missing or making the field goal, and converting or getting stopped on fourth down).
There’s not nearly the time or space here to get into details of how the leading analytics firms and teams calculate win probabilities.
But my main question is whether they could be confident they had the right data and assumptions in their algorithms. After all, the economy melted down in 2008 in part became highly paid analytics experts (“quants”) on Wall Street made catastrophic errors in the assumptions in their algorithms for the risks of home mortgages defaulting.
So I asked NFL analytics expert Frank Frigo of EdjSports and Eric Eager of Pro Football Focus, whose companies contract with NFL teams on analytics matters including win probability. Both said teams have asked essentially the same question during sales pitches and consultations. How do you know how good the quarterback we’re facing is? Or the defense? Or the weather? Or any other numerous factors?
Both said their algorithms can be customized for weekly matchups, based on data for the teams’ offenses, defenses and special teams. This is far more sophisticated than the websites that have simple win-probability calculators, which are based on NFL play-by-play data and account for only score, time of game, down and distance, and the Vegas line.
You might remember seeing those calculators saying that in the third quarter of Super Bowl LI, New England had only a 1 percent chance of coming back to beat Atlanta. They obviously failed to account for, among other things, Tom Brady.
The truth is there’s no way to be sure how exact analytics are in assessing win probabilities. But there’s no doubting they’ve improved dramatically in the last few years as experts have learned from coaches to take more factors into account.
“In football there’s a lot of data,” Frigo said. “We have a much better feel for the distribution of outcomes and the relative strengths of opponents.”
NFL rules prohibit teams from calculating win probabilities during games, so they (or firms they commission, such as EdjSports and Pro Football Focus) create books of charts that can be customized for each week’s opponent and weigh hypothetical decisions, such as whether to go for a fourth down based on down and distance, field position and time of game.
Though teams are beginning to embrace analytics more, Frigo and Eager said one area where they consistently still see mistakes is on fourth downs. For instance, analytics tell them teams should go for it on fourth-and-one at almost any field position early in games. As you get closer to midfield and beyond, the analytics say most teams are often better off going for it even on fourth-and-4 rather than punting, depending on the time of the game.
“The problem is there’s a level of regret when you fail that is really hard to swallow,” Frigo said. “You might be fourth-and-one on your own 30, an average NFL team might be 70 percent-plus to convert it, which is a huge gainer to retain possession and get a new set of downs. But the risk of giving up possession (there) can be too much for people to swallow.”
Said Eager: “It’s about winning football games, it’s not about not losing football games. I think that’s where coaches get themselves in trouble.”
Which gets us back to LaFleur’s and Rivera’s decisions last week.
LaFleur was ahead by four points when he had to decide whether to go for the touchdown from the 1 or take the easy field goal on the final play of the first half. Carolina stuffed running back Jamaal Williams, and LaFleur took a lot of abuse from some fans and media for not taking the sure three points.
EdjSports’ algorithm showed the Packers’ win probability was 1.5 percentage points higher going for the touchdown rather than the field goal — they had a 90.2 percent win probability by going for it, and 88.7 percent if they’d kicked the field goal. There’s a more thorough explanation of the math at the end of this column, but the numbers clearly say going for it was the right call.
“It is only a 1.5 percent (gain) because the Packers’ (win probability) is already north of 90 percent,” Frigo said, “and it is hard to make substantial gains on a single decision at that level with a full half of football remaining.”
Eager explained it differently and said that by kicking the field goal the Packers’ win probability actually would have gone down by 4 percentage points (because of the opportunity cost of not going for a touchdown from only one yard out). It would go down more (8 percentage points) by failing to get the touchdown, but it would go up 7 percentage points by scoring the touchdown. Risking 4 percentage points to gain 7 was the right call.
He said another way to look at it was, the conversion rate on fourth-and-goal from the 1, which is essentially what LaFleur faced because there were only two seconds left in the half, ranges from 55 to 65 percent, depending on the quality of the quarterback and offense. So going for it is was worth at least four points for the Packers (seven points times a 60 percent success rate). The odds of making the field goal were nearly 100 percent, so kicking was worth three points. That means attempting the field goal rather than going for it would have surrendered a point.
“The benefit of (an) 11(-point lead) far outweighs the benefit of (the field goal),” Eager said.
Rivera’s decision was clear cut. Down 14 points late in the game you should always go for two on your next touchdown. Two-point attempts are successful about 47 percent of the time, so odds are you’ll convert at least one of the two attempts if you score two touchdowns. If you convert the first one, you can win the game with a second touchdown and extra point. If you don’t convert the first, you can still tie with a two-pointer on the second.
Both coaches made aggressive calls at key times Sunday. LaFleur went with his gut, Riverboat Ron with his head. Both were right in an NFL where conventional wisdom is starting to give way to analytics.
Here is a more detailed explanation from EdjSports of how it concluded the Packers were better off going for the touchdown on the last play of the first half against Carolina last Sunday:
There are two parts to this assessment. First, the EdjSports model simulation is in favor of going for it by 1.5% GWC (game-winning chance). This is a function of the game state and the relative strengths and weaknesses of the teams. This is advantageous to the Packers as they are top five in rush and pass analytics, and Carolina ranks poorly defensively.
To provide perspective as indicated in the table above, the key reference point is the conservative decision to kick the field goal. The Packers are 88.7% GWC if they decide to kick. Keep in mind they were a substantial pre-game favorite (73% ) and will be leading by seven at the start of the second half with the first possession. To understand the risk and reward, we must compare the successful TD attempt and unsuccessful TD attempt to the FG attempt. A successful TD boosts the Packers’ GWC to 94.1 percent and a failure drops it to 82.7 percent. The Packers stand to gain 5.4 percent and risk 6 percent on the TD attempt. The required success rate to make it worth going for the TD is then 6/(6 + 5.4) or 52.6%. The expected success rate of an average NFL team in this situation is reasonably between 55 percent to 65 percent. The comparative team power indexes place the Packers in the higher end of this range.
Again, it is only a 1.5% difference because the Packers’ equity is already north of 90 percent and it is hard to make substantial gains on a single decision at that level with a full half of football remaining.