Machines are tackling the NFL

A model that can predict the unpredictable New England Patriots

It’s said that familiarity breeds contempt in personal relationships. In the NFL, it might also breed predictability. Although the New England Patriots and their coach Bill Belichick are often called unpredictable, it turns out that machine learning models are actually pretty good at guessing what they’ll do.

Alex Tellez, who works for machine learning startup H2O, built a model he says can predict with about 75 percent accuracy whether the Patriots will run the ball or pass it on any given play. He used 13 years of data — all available on NFL.com — that includes 194 games and 14,547 plays. He considered a dozen variables for each play, including things such as time, score and opposing team.

Tellez thinks it might be possible to build a model that predicts plays with even more accuracy. He noted, while slyly touting his company’s software, that this one was created with just a few clicks using the H2O platform. Spending more time and tweaking some of the features might improve accuracy, and he suggested that feeding data into a recurrent neural network (which would have some ability to remember some results from one play to the next) might help account for the emergence of players like running back Legarrette Blount, who can skew play-calling in the short term.

belichick

Tellez’s model works, in part, because of how long Belichick and quarterback Tom Brady have been together — 15 seasons now. That’s a lot of time to amass data about what types of plays the team will call in any given situation, with at least two constant — and important — variables in the coach and the quarterback.

“Realistically, Bill Belichick and Tom Brady, those are like the only dynamic duo,” explained Tellez. “You couldn’t do it with the Raiders,” he added, alluding to that team’s revolving door of coaches and quarterbacks.

Or even the New England Patriots’ Super Bowl competitor, the Seattle Seahawks, who are working with a fifth-year coach and third-year quarterback.

Last year, I wrote about Brian Burke, the founder of Advanced NFL Analytics and the guy whose models power the New York Times 4th Down Bot. “The number of variables, it explodes geometrically,” he said about the challenges of predicting football plays.

belichick2

Still, even if predicting the likelihood of a run or a pass remains an unsolvable challenge for most of the NFL, the proliferation of data has already and likely will continue to change the face of football — and sports overall — in some very significant ways. Some obvious ones are the advanced metrics used by Major League Baseball teams to rate players beyond just their batting averages or earned-run averages, the now-trite “moneyball” method building rosters, and the remarkable success of expert statisticians such as Burke and FiveThirtyEight’s Nate Silver.

At our Structure Data conference in March, data executives from ESPN and real-time player-tracking specialist STATS will discuss how access to so much data is changing the fan experience, as well, and even the on-court decision-making in sports such as professional basketball.

Depending on whether anyone can build accurate-enough models, Tellez actually suggested we could see live sports broadcasts include predictions of the next play similar to how ESPN predicts outcomes in its World Series of Poker broadcasts. While his Patriots model took about 30 seconds to run, live-broadcast models would have the benefit of being able to pre-load data for the specific game situation and only run against that data, he said.

Richard Sherman
Richard Sherman

He also has another idea for applying advanced data analysis the NFL — predicting rookie performance in the NFL combine. That’s where draft prospects go to show off to NFL scouts how big, fast and strong they are. However, not all prospects participate in all the events, which can give teams an incomplete view of their athletic prowess.

Tellez built a special type of neural network, called a self-organizing map, to analyze all combine performance for cornerbacks, specifically, and then fill in the blanks when players opt to skip a particular exercise. Think about it like Google’s Auto-Fill feature, which predicts missing values in spreadsheets. He says he discovered that good 40-yard dash, shuttle run and 3-cone times tend to correlate with high draft picks and future success, so being able to predict those times even if a prospect doesn’t do them could be valuable.

Of course, Tellez noted, stats don’t always tell us the truth. His model, as well as NFL scouts, predicted Seattle Seahawks cornerback Richard Sherman as a mid-round draft pick. The Seahawks drafted him in the fifth round. He’s now considered one of the league’s most-dominant cornerbacks and most-recognizable players.

6 Responses to “A model that can predict the unpredictable New England Patriots”

  1. Taghkanic

    Very interesting. However, I wonder whether using such data would actually help in the midst of a game. Being able to predict run or pass on any given play 70-75% of the time sounds great. But if a defense goes all-in on each prediction—rather than relying on split-second analysis and instincts at the snap—what happens on the 25-30% of the plays on which you are wrong?

    If that missed prediction leads to big plays, that could actually lead to worse performance. If a team hedges on each play, setting up for either a run or a pass, it might allow more total yards but fewer big plays.

    As the Patriots themselves have shown, the total number of yards allowed is not necessarily the key predictor of who wins. The Pats’ “bend don’t break” approach has for years meant limiting opponents to field goals where others allow touchdowns, or no points at all after a long drive. It doesn’t matter if the other guys grind out 50 yards, only to come away with 0 or 3 points.

    Games can break wide open, however, on just a couple of blown coverages. So I again wonder if such predictions can actually help, as a fundamentally sound team is still going to want to protect against both the run and the pass on every play.

    A poker analogy might be appropriate here: Winning 10 small hands isn’t much use if you go on to lose one giant hand, which erases those wins and more.

  2. lizzzy321

    LOL This is the funniest thing I’ve read all week. So what you are basically saying is that the Patriots are the unbeatable team, that you need software to make a game plan to have a chance of beating them? So if you beat the Patriots using software, who gets the win, the team and their coach, or the software company. LOL

  3. Joe Appelbaum

    Is 75% really all that good? Using downs DJ distance, score, formation and time remaining I’d wager that many interested observers would come close to that figure. It would be much more interesting (and useful) to see the rate on 1st and 10 in a close game.

    • With the amount of information available 75 percent is bad. Just saying Pass for every play will get you to 60 percent. Then put in the no brainer situations like 3rd and 4th and 1 or 2nd and 3rd and 7-15 yards and you are going to be close to 70 percent right there. Last 2 minutes of a half score gets you another couple of percentage points.

      • If you read the article, it said this was built in minutes with currently available data set. Imagine what one could do with lots of time, all data available, and stats on specifically 3rd/4th down play calling, along with factoring in the time of the game at which the play is called and what the current score is. Possibly looking at how good the quarterback is at running the ball himself in Wilson’s case, versus a pass.

        No one said that all these factors were put in this model. But someone should build one.