In the six months Carson Wentz has been a Washington Commander, the discussion of what people should expect from him this season — Will he resuscitate his career? Advance the Ron Rivera era? — has highlighted a divide common in the analytical age.
Wentz’s variance — his penchant for wow throws and ugly blunders — prompts a bigger question: What is the best way to evaluate a quarterback statistically in 2022?
Though data will never capture everything, especially in a sport as complicated and opaque as football, it can provide a critical cross-check for what we will see in Sunday’s season opener against Jacksonville and throughout 2022. Still, in the sea of analytics, it’s often difficult to determine which measures offer the best signal amid the noise.
“There’s no one statistic where I look at that and say: ‘That’s it. That perfectly ranks the quarterbacks,’ ” Pro Football Focus analyst Eric Eager said, adding that it’s important to consider many factors. “There’s just so many differences in how they’re playing, and there’s so many differences in what wins on even a given day.”
In football, stats exist in three buckets. Outcome reflects only results and accounts for the lion’s share of the stats universe. Charting adds a schematic lens — think passing yards on certain route concepts — but is limited without firsthand knowledge of the play call. Tracking measures movement, such as player speed or throw spin rate, and offers tremendous insight but is the least accessible.
From 1933, when the forward pass was legalized, until about a decade ago, fans mostly relied on the basic outcome stats, such as passing yards, or the popular-but-deeply-flawed passer rating. That stat leans heavily on touchdown-to-interception ratio, which means it ranks Wentz higher (13th) than an all-encompassing metric that includes sacks would.
In the early 2010s, ESPN took advanced stats mainstream by introducing a new metric called total quarterback rating (QBR). This started nudging the public “out of the dark ages,” said Ben Baldwin, a football data scientist and contributor to the Athletic. As the analytics community grew online and on Twitter — and more and more metrics emerged — analysts and academics have tried to untangle a quarterback’s play from his team’s and quantify it.
Ultimately, most of the analytics community has coalesced around the metric expected points added (EPA), which gives yardage context by delineating between, say, a three-yard gain on first and 10 and on third and two. Analytics-minded observers have also come to value completion probability over expected (CPOE), a fairly predictive metric that uses several stats — field position, pass location and whether the quarterback was hit on the play — to account for the difficulty of attempts.
A few years ago, Baldwin decided to mitigate the weaknesses of EPA (which is unable to separate the quarterback from the rest of the offense) and CPOE (which can’t capture a quarterback’s ability to make throws easier before actually throwing) by adding them together. EPA+CPOE is now generally regarded as one of the best summation stats for quarterback play.
“Without access to tracking data or going through and watching every single play, I think this is probably close to the best [outcome analytics] can do,” Baldwin said.
Last year, of the 34 quarterbacks with at least 250 plays, Wentz ranked 21st in EPA+CPOE.
But there are even narrower indicators by which to judge quarterback play. Derrik Klassen of the website Football Outsiders thinks the central question is “Who is the best player to build around?” rather than “Who is most productive in the confines of his system?”
To isolate talent, Klassen looks at “true dropbacks,” a charting stat that excludes plays with motion, play action, screens and run-pass options. Though the gadgetry helps quarterbacks, Klassen believes it’s easier to adapt to such tools than it is to succeed without them. In true dropbacks, he said, the offense isn’t trying to fool anyone. The play-caller is asking the quarterback to win “on hard mode.”
“[True dropbacks are] just like the ultimate ‘put up or shut up,’ ” Klassen said. “It’s the stuff that you pay for.”
Last season, Wentz had 224 true dropbacks, 10th most in the NFL, according to the website Sports Info Solutions. He threw shorter and off-target more often than his peers and took sacks at a high rate. Of 33 qualified passers, he ranked mostly between 21st and 25th, the lower end of average.
One of the biggest criticisms of Wentz’s game during his career is his tendency to extend plays and take bad sacks — because “there’s a lot of evidence that quarterbacks, to a great extent, control how often they get sacked,” Baldwin said. In the second preseason game this summer, Wentz took a third-down sack that pushed the Commanders out of field goal range. This habit also helps explain why charting metrics are wary of Wentz; PFF graded him 23rd among quarterbacks last year.
At Commanders headquarters, Coach Ron Rivera is skeptical of many stats and likes the phrase, “Figures lie and liars figure.” Offensive coordinator Scott Turner does consult analytics to help develop the game plan, but the quarterbacks themselves don’t use them to assess their play.
Taylor Heinicke said it’s important to identify certain concepts or throws to work on but added that stats can “overcomplicate things” and limit his ability to play fast and free. Wentz said he evaluates himself exclusively with film — and he re-watches every game the same day it’s played, sometimes not finishing until he’s in bed and about to go to sleep.
“Stats are going to lie to you,” he said. “You just [have to think]: ‘What could I have done better? Where could I be more accurate or [have] better decision-making?’ . . . It’s truly, strictly on the film.”
In his film review of Wentz’s preseason, Hall of Fame quarterback Kurt Warner said Wentz showed several positive signs — good processing, comfort in the Commanders’ scheme — but that old, recurring issues also remained. He noted that Wentz’s inconsistent mechanics and footwork regularly caused him to miss easy throws.
This season, Washington’s coaches will pay extra attention to a few specific stats. Last year, Washington completed 32.4 percent of its passes deeper than 20 air yards, 23rd in the NFL, and it acquired Wentz in part because his arm strength should open up throws downfield. Overall, Wentz has completed 36.3 percent of such passes, 18th among active quarterbacks.
Even if Wentz raises Washington’s hit rate on deep throws by only a few points, Rivera said, it would be a big boost to help open up the rest of the offense.
“We’ve got to make sure that we’re giving him every opportunity to succeed,” he added. “We’re doing that.”
This fall, as Wentz tries to stabilize his career, the ways in which we understand his play may continue to evolve. In January, the NFL introduced a new metric called passing score, which incorporated machine learning and tracking data. The league didn’t publicly release a leader board for the 2021 season but hinted it may in the future.
In the meantime, the ability to cross-reference Wentz’s performance with statistics will remain insightful but imperfect. Analysts see Wentz as having the tools to succeed — a forward-thinking coordinator and talented supporting cast — but an uninspiring track record to suggest he will.
Eager, of PFF, said what comes next is the most fun part. Players can defy expectations, and if they do, it’s up to analysts and fans to dig into the data to puzzle out how they did it.
“That’s what makes football statistics great,” he said.