The Shifting Wind

To even the least analytics-minded observer, something seemed different during Week 16 of the 2021 NFL season. Coaches appeared to be going for it on fourth-and short-routinely, often in their own territory, and yet commentators’ heads weren’t exploding. Has the thinking really shifted, and is it here to stay?

There were approximately 100 non-trivial fourth-down decisions that occurred during Week 16. “Non-trivial” is the subset that eliminates obvious late-game desperation attempts or, to be more specific, the type of decisions NFL coaches have always gotten right. EdjSports analyst Ian O’Connor, in his weekly fourth-down report, identified 35 correct fourth-down go attempts. This is the highest count since we have been keeping records and may well be the highest count for a single week in NFL history. Of the 65 decisions that were deemed to be suboptimal, many fell into the gray area of low confidence, and only 16 resulted in a net cost of greater than 2% Game-Winning Chance (GWC). Perhaps we are close to reaching herd immunity.

While there is clearly a visible change in behavior this season, there are still many indicators that suggest we have a long way to go to reach the type of enlightenment that machines have introduced to so many other competitive games. I strongly suggest watching the “AlphaGo” documentary to get an idea of what I am talking about. Bill Belichick may be the most revered coach in NFL history, yet he sits near the bottom of the EdjSports Critical Call Rankings. His aggressiveness on fourth downs began to regress around 2011, and he has not managed to keep up with his more analytically open-minded peers in recent years, at least in this specific category of coaching. Even this weekend, as we witnessed all this progress on fourth downs, Matt Lafleur managed to squander -12.4% GWC on a single decision late in the contest against the Browns. With a fourth-and-6 on their own 42-yard line and clinging to a two-point lead with 2:14 remaining in the game, the Packers punted the ball away to the Browns. On the subsequent drive, a Baker Mayfield interception on third down from midfield kept the Browns from exploiting Lafleur’s error in judgment. To be fair, this is the type of highly-leveraged fourth-down decision that almost every NFL coach would still get wrong.  It defies all the commonly held precepts that have governed such decisions for decades. 1) The Packers were leading in the game; 2) the ball was in their own territory; and 3) it was a long fourth-and-6. Yet this was, by our extensive analysis, the costliest decision of the week.

There is still much room for improvement, but we have come a very long way. When Chuck Bower (an experimental physicist and one of the founders of EdjSports) and I first presented our model and revelations to an NFL team, it was January of 2004. We had begun working on the fourth down simulation model in 2001 and finally managed to get an audience with Marvin Lewis and the Cincinnati Bengals’ front office.  Unfortunately, we were met with cynicism, apathy, and patronizing well wishes. Motivated by other independent research and the innovative metrics and analytical insights of Aaron Schatz, we pressed on and were ultimately successful in contracting with more than one-third of the NFL teams over the next 15 years, including two Super Bowl champions. Because of the complex nature of NFL franchises, shaping the structure of analytics departments is not always the same as extracting benefit. With few exceptions, our tool was treated more as a novelty than a game changer. There has always been a big difference between awareness and implementation when it comes to NFL coaching. However, this season marks a satisfying, and long overdue, tipping point toward adoption.