Over the past month, thirty-two NFL teams have been reduced to two for the ultimate prize in football. How well did the current league rules find the best/most deserving contestants? Edj Analytics has analyzed the full-season performances of all 32 NFL teams from all 266 (regular season and playoff) games leading up to Super Bowl 50 And the results indicate that the Broncos and the Panthers are statistically deserving of their spots in the big game this Sunday.
We’ve used the Edj Analytics football engine to evaluate every non-special teams play of the NFL 2015 season. Using a game simulator, the result of every play during the 2015 season is compared to how well a league-average team would have done in that particular situation, and with that particular playcall against the actual opponent. Offensive and defensive performances— separated into passing plays and running plays— are tallied for each team. The end result for each game is then synthesized into four numbers for each team: offensive pass, offensive rush, defensive pass, and defensive rush.
For each team, these four quantities are summed over all games for the season. (The typical stats such as field goal success rates and kick return distances are used to grade the special teams’ plays.) We then utilize a simulation engine to perform a round-robin tournament, pitting every (customized) NFL team against every other. Thanks to today’s computing power, we are able to simulate each contest (team A vs. team B) 400,000 times. The final result is a top to bottom ranking of the 32 teams based on total wins in the tournament.
While listing the teams in exact order of their final total number of wins seems intuitive, the data doesn’t account for the uncertainty in the method. However, clear breakpoints in the results are sufficiently insensitive to the methodology, allowing us to group them. Those results, from top to bottom, alphabetized within a group, are:
Group 1: Arizona, Carolina, Kansas City, New England, NY Jets, Seattle.
Group 2: Cincinnati, Pittsburgh.
Group 3: Buffalo, Denver, Green Bay, Minnesota, Washington.
Group 4: Atlanta, Detroit, Houston, Indianapolis, New Orleans, NY Giants, Oakland, Philadelphia, St. Louis, Tampa Bay.
Group 5: Chicago, Miami.
Group 6: Baltimore, Cleveland, Dallas, Jacksonville, San Francisco.
Group 7: Tennessee.
Group 8: San Diego.
13 teams have fallen into our top three “elite” groups, however, the NFL only allows 12 into the playoffs. Playoff team Houston is notably absent from our top groups, while the Jets and Buffalo made the cut, even though they both missed out on post-season play. Of these three disparities, the biggest surprise is Buffalo with their 7-9 record, being among the elite.
Having to play four games against their two division foes in Group 1 helps understand their location in our evaluations. With only 13 regular season opponents chosen from 31 possible, “strength of schedule” plays a role. The illustration here (insert now) shows how each team compared to each of their 31 peers. Again, when the results were close we factored in our method’s uncertainty to break the comparisons into five categories, from clearly superior to clearly inferior.
Certainly wins (and losses) in actual competition are the best measure of a team’s success, and should be the primary goal of professional competition. By measuring performance of every non-special team play we give value to instances where the W/L result was already decided. Denver has reached their penultimate goal yet only landed in our 3rd group. Comparing the two finalists’ scores we note that of Denver’s 14 wins, only three were by double digits. Carolina on the other hand outscored their opponents by double digits 9 times. Is it possible that some of Carolina’s opponents eased up when far behind, while Denver’s played tough to the end, given how close those games were? Regardless, no one will argue that the Broncos won the games that mattered, particularly their last two against teams (Pittsburgh and New England) positioned higher than they in our rankings.
Unfortunately, injuries take their toll in our simulation results, just as they do in real life. For example, Pittsburgh’s games which were played without Ben Roethlisberger were weighed equally with the games in which he played. San Diego’s cellar position also can attributed to the unusual number of injuries they suffered this year.
Despite its imperfections, we draw satisfaction in our rankings from the fact that there is a very strong correlation between Win/Loss records and the ordering of our results. In addition, the NFL’s method of determining the playoff teams holds up pretty well. Unfortunately for Jets and Bills fans, that process has its own imperfections, and those turn out to be quite costly to these two worthy franchises.