A model that can predict the unpredictable New England Patriots

It’s said that familiarity breeds contempt in personal relationships. In the NFL, it might also breed predictability. Although the New England Patriots and their coach Bill Belichick are often called unpredictable, it turns out that machine learning models are actually pretty good at guessing what they’ll do.

Alex Tellez, who works for machine learning startup H2O, built a model he says can predict with about 75 percent accuracy whether the Patriots will run the ball or pass it on any given play. He used 13 years of data — all available on NFL.com — that includes 194 games and 14,547 plays. He considered a dozen variables for each play, including things such as time, score and opposing team.

Tellez thinks it might be possible to build a model that predicts plays with even more accuracy. He noted, while slyly touting his company’s software, that this one was created with just a few clicks using the H2O platform. Spending more time and tweaking some of the features might improve accuracy, and he suggested that feeding data into a recurrent neural network (which would have some ability to remember some results from one play to the next) might help account for the emergence of players like running back Legarrette Blount, who can skew play-calling in the short term.

belichick

Tellez’s model works, in part, because of how long Belichick and quarterback Tom Brady have been together — 15 seasons now. That’s a lot of time to amass data about what types of plays the team will call in any given situation, with at least two constant — and important — variables in the coach and the quarterback.

“Realistically, Bill Belichick and Tom Brady, those are like the only dynamic duo,” explained Tellez. “You couldn’t do it with the Raiders,” he added, alluding to that team’s revolving door of coaches and quarterbacks.

Or even the New England Patriots’ Super Bowl competitor, the Seattle Seahawks, who are working with a fifth-year coach and third-year quarterback.

Last year, I wrote about Brian Burke, the founder of Advanced NFL Analytics and the guy whose models power the New York Times 4th Down Bot. “The number of variables, it explodes geometrically,” he said about the challenges of predicting football plays.

belichick2

Still, even if predicting the likelihood of a run or a pass remains an unsolvable challenge for most of the NFL, the proliferation of data has already and likely will continue to change the face of football — and sports overall — in some very significant ways. Some obvious ones are the advanced metrics used by Major League Baseball teams to rate players beyond just their batting averages or earned-run averages, the now-trite “moneyball” method building rosters, and the remarkable success of expert statisticians such as Burke and FiveThirtyEight’s Nate Silver.

At our Structure Data conference in March, data executives from ESPN and real-time player-tracking specialist STATS will discuss how access to so much data is changing the fan experience, as well, and even the on-court decision-making in sports such as professional basketball.

Depending on whether anyone can build accurate-enough models, Tellez actually suggested we could see live sports broadcasts include predictions of the next play similar to how ESPN predicts outcomes in its World Series of Poker broadcasts. While his Patriots model took about 30 seconds to run, live-broadcast models would have the benefit of being able to pre-load data for the specific game situation and only run against that data, he said.

Richard Sherman
Richard Sherman

He also has another idea for applying advanced data analysis the NFL — predicting rookie performance in the NFL combine. That’s where draft prospects go to show off to NFL scouts how big, fast and strong they are. However, not all prospects participate in all the events, which can give teams an incomplete view of their athletic prowess.

Tellez built a special type of neural network, called a self-organizing map, to analyze all combine performance for cornerbacks, specifically, and then fill in the blanks when players opt to skip a particular exercise. Think about it like Google’s Auto-Fill feature, which predicts missing values in spreadsheets. He says he discovered that good 40-yard dash, shuttle run and 3-cone times tend to correlate with high draft picks and future success, so being able to predict those times even if a prospect doesn’t do them could be valuable.

Of course, Tellez noted, stats don’t always tell us the truth. His model, as well as NFL scouts, predicted Seattle Seahawks cornerback Richard Sherman as a mid-round draft pick. The Seahawks drafted him in the fifth round. He’s now considered one of the league’s most-dominant cornerbacks and most-recognizable players.