Remote handy, notebook at his hip, Jim Caldwell spends fall weekends in his man cave, saddled to his leather couch, eyes glued to a 100-inch television that’s often split into four screens so he can watch every possible college football and NFL game.
The NFL loves ‘quarterback whisperers.’ Why doesn’t it love Jim Caldwell?
As NFL owners increasingly turn to young, White offensive ‘gurus,’ the coach who mentored Peyton Manning, Matthew Stafford and Joe Flacco is out of a job.
These aren’t the afternoons of some obsessive fan or rocking-chair retiree.
“I’m working,” Caldwell, 67, says with a laugh.
The fact that Caldwell spends Sundays at home in Lewisville, N.C., watching football on television suggests there is a weird amnesia among people who hire NFL head coaches. They go chasing after the latest “quarterback whisperer” and elevate young White men above their capacities, apparently insensible to the fact that it was a Black coach who altered the career of arguably the most intelligent quarterback in the history of the league, Peyton Manning, and who raised the games of two eventual Super Bowl winners, Joe Flacco and Matthew Stafford.
The NFL has spent more than a century struggling to embrace Black men as leaders — first in helmets and now in headsets. A recent analysis by The Washington Post found that Black coaches receive iniquitous treatment in both hiring and retention: They face narrower paths to the job than their White counterparts and are fired more quickly even when they win more games.
Caldwell’s personal experience is human evidence of the numbers. He is one of five rookie head coaches to ever reach a Super Bowl, in his first season at the helm of the Indianapolis Colts in 2009. Yet he would be fired in only his third year in 2011, assigned the blame for a single disastrous losing season when Manning sat out for neck surgery. A similar experience followed with the Detroit Lions from 2014 to 2017: Caldwell won at least nine games in three of his four seasons and made the playoffs twice, posting the highest winning percentage of any full-time Lions coach in the past 50 years. Yet he was fired there, too. Caldwell’s dismissals in Indianapolis and Detroit would be cited prominently in the class-action discrimination lawsuit filed by Brian Flores against the NFL in February 2022.
Caldwell has not received a head coaching opportunity since 2017. Two Super Bowl trophies sit atop the bar in his man cave, twin homages to his winning influence. How many franchises — especially those in Arizona, Denver and Chicago — could use his sideline maturity and proven record of elevating not-yet-great quarterbacks or reviving stagnant ones?
In 2002, Manning was not yet the all-time great he would become. He had just finished his fourth year in the league, and his record was 32-32. Though he was obviously one of the most potent young players in the game — he had led the team to 13 wins in 1999 — he was also a league leader in interceptions. As a rookie, he had set a season record for them (28) that still stands. Heading into his fifth year, he was coming off a 6-10 record in which he had thrown 23 more interceptions and added seven fumbles.
“It was like, who am I going to be?” Manning recalls.
The Manning years
Manning’s work with Caldwell from 2002 to 2006 was a transformative phase in the quarterback’s career. How many scores of gifted NFL quarterbacks never break out of unnoticed physical patterns, over-rely on what they do best and fail to see what they do worst? And remain stuck in a cycle of middling success? What would Manning’s record have been had he not fallen into the hands of a coach so technically knowledgeable and worthy of his talent?
In 2002, Tony Dungy, the only Black coach in the Pro Football Hall of Fame, took over as head coach of the Indianapolis Colts and brought Caldwell with him as his assistant head coach in charge of quarterbacks.
The record book reflects a major difference between who Manning was before Caldwell (and Dungy) and who he was after. He reduced his interception rate by half by his fourth season with them. The Colts won at least 12 games for six straight seasons after the coaches’ arrival, peaking with a Super Bowl title. To get to that point, however, involved a certain amount of correction.
Manning’s performative U-turn was a credit to Caldwell’s ability to fundamentally improve a still-young player who was at once one of the most brilliant scanners of the field — “near genius level,” Caldwell says — and yet a flawed and burdened performer. Press critics suggested that Manning, the No. 1 overall selection in the 1998 draft, was not quite living up to the rookie contract he had signed worth $48 million. Asked once what he intended to do with the money, Manning replied, “Earn it.”
On the day that Dungy arrived in the Indianapolis offices, Manning came by to say hello. “I want to be coached,” Manning told him. “I want to win.” He brought to Dungy’s office a list on a yellow legal pad. “These are the things I want to get better at,” he said. Manning knew he needed help.
The first thing Dungy and Caldwell told Manning was that he threw way too many interceptions. “That was the one area we had to be certain we could see some improvement,” Caldwell says. He decided to go back and review every single one Manning had thrown over the previous four years. He made a list of them, all 81, and typed it up. Then, Dungy and Caldwell convened a meeting with Manning. It would become a ritual with them: from then on, every offseason began with a review of Manning’s interceptions.
“It was a humbling start to the offseason,” Manning recalls.
Dungy and Caldwell handled Manning carefully. Before they could really help him, they had to understand what was going on in his head, the context in which he was operating.
They began by asking questions. What was Manning’s intention when he threw the ball? Why did he throw it there? The quarterback had a specific reason for every misdirected pass. They were mistakes of commission, not omission, and Manning took full responsibility for every one.
“Every interception has a story, but nobody wants to hear it, right?” Manning says. But Caldwell and Dungy needed to hear it. It turned out Manning felt pressure to carry the team by making big plays because the Colts were so bad on defense, having given up more points than any other NFL team the previous season. His interceptions were calculated gambles based on the situation and scoreboard.
They would look at the film, and Manning would say, “I did that because if we were going to win the game, we’ve got to go for it there, I’ve got to throw a touchdown pass. It was third down, and I didn’t want to punt.”
Manning hated to punt. The Colts defense was so weak that if they punted, it might be six minutes before he got the ball back.
“He felt he had to make every throw,” Dungy recalls. “Every tough throw, he could explain why he made it. He wasn’t making reckless decisions. It was what he thought was right.”
Dungy promised to fix the defense. In the meantime, Caldwell tried to get across to Manning that sometimes a better decision was a more discreet one: checking down to a shorter throw to let a teammate try to make a play or even throwing the ball away. Or even, yes, punting and seeking better field position.
In the next season, the Colts went 10-6. Still, Manning threw 19 more interceptions, third most by any quarterback in the NFL.
That March, on the very first day of the offseason, Caldwell and Manning met at 7 a.m. Caldwell had already spliced up an interception tape. Together, Manning and Caldwell studied each throw that landed in the hands of the opponent and graded it as “A” or “UA,” for “Avoidable” or “Unavoidable.”
“ ‘Unavoidable’ is you hit the receiver right in the chest and it tips up and the linebacker gets it,” Manning says. “You’d make the same throw tomorrow or next week all year. It’s just the way it worked out. The ‘avoidables’ were ones where you had the running back wide open but you’re forcing one down the sideline to Marvin Harrison in between a safety and corner and it’s intercepted. That’s an undisciplined throw, and that’s avoidable.”
When that tape ran out, Caldwell cued up a new one, of all the close-call interceptions that the defender could have had if he hadn’t dropped the ball or blown the coverage. “It was kind of a hidden tape that a lot of people don’t get to,” Manning says. They graded those throws, too. Next, they watched tape of all the times he held the ball too long. Then they watched tape of the potential touchdown passes that he had missed, when he should have scored but for some reason didn’t.
By then Manning was beginning to realize that he had met a coach who was as analytical as he was and moreover who had a technical knowledge that could help him clean up his play. “Me and Jim Caldwell were built very much the same way,” Manning says. Caldwell agrees.
“We were working on getting him better at the things that maybe, perhaps, he didn’t do as well, you know, as he thought,” Caldwell says. “Now he’s a perfectionist, right? So he just devoured that information and loved it. It’s not very often that you find a guy who is as accomplished as he is that really wants to get into the details, the nuts and bolts of fundamental football.”
They studied every lousy, failed play, to see what was correctable. “What we did off that film, whether it was the interception tape or the sack tape, was look for commonalities,” Manning says. Caldwell noticed something. “It seems, Peyton, like you throw a lot of interceptions when there’s a defensive lineman at your feet,” he said. “You can’t finish the throw; the ball is sailing on you.”
It was one thing to notice it. It was another to do something about it. What impressed Manning more than Caldwell’s analytical eye was that he could create drills to cure his nervous feet. When they went on the practice field, Caldwell began hurling heavy bags at Manning’s cleats.
“He’d throw a blocking bag at me like a lineman and see if I could still make the throw,” Manning says. “So, there was very much of a process: Let’s take what we see on film and put it on the field. Otherwise, what are you watching the film for? If you’re just saying, ‘Oh those were a lot of avoidable interceptions,’ that doesn’t help. You better go do something about it on the field. So that’s when I felt like I kind of got going with them. And I had my best season to date, and shared MVP, and we won our first playoff game.”
The pattern repeated itself over and over. Caldwell would notice a physical nuance in Manning on tape and then create methods to correct it on the field. As a young coach, it had frustrated Caldwell that most film was taken from an overhead perspective, usually from a high tower in the end zone. Instead, Caldwell had an aide film Manning in practices from small step ladders placed at various angles just behind the QB, so that he could see action from the player’s perspective.
“To capture the things that quarterbacks were seeing from his vantage point,” Caldwell says. “Then also we could look at all of his mechanics. … It was something a little bit different than what he had done or been exposed to early in his career.” During a sprint-out play, for instance, Caldwell noticed that Manning struggled to pull up and make an accurate throw under duress because of shoulder mechanics. “So we would attack his upfield shoulder, make him pivot his feet, get his body in position to throw an accurate ball,” Caldwell recalls.
One of Caldwell’s drills in particular turned out to be a difference-maker for Manning and the Colts. Caldwell was a stickler for practicing safe ball snaps, and he drilled Manning and Jeff Saturday, the team’s center, endlessly. Before practice even started, he’d put the two through a full-speed series of exchanges, “and it was a sight to behold,” Manning recalls, “full cadence, audibles, changing the plays, creating all kinds of oddball difficult scenarios, and it’s full speed, a full lather.”
Caldwell worked on Manning’s and Saturday’s movements; he wanted them to be able to recover and work together comfortably in any situation, even if the snap hit the dirt. “It’s on the ground for one reason or another, and it dribbles back to the quarterback. And then what do you do from that point?” Caldwell asks. He instituted what he called “the hot drill.” He’d toss a ball to Manning’s left and make him shuffle sideways, recover and then get the ball away to a receiver. Another ball would go flying to Manning’s right or over his head. And every third week or so, Caldwell would spray the balls with water until they were dark and slick and make the offense practice with those.
In the 2006 season’s Super Bowl near Miami, an inch of rain fell. Manning and Saturday never mishandled a snap all night. Meanwhile, their opponents, the Chicago Bears, lost two fumbles on the snap. Turnovers were the difference in the game, as the Colts went on to a 29-17 victory.
“There is a difference in guys understanding X’s and O’s and being able to teach you the techniques and fundamentals of how to accomplish that in X’s and O’s,” Saturday says. “And what I’m telling you about Caldwell is that he understood the techniques and fundamentals required to get player where they need to go. You know, when you draw anything up on the board, it can look pretty good, right? Like: ‘Oh, wow. Conceptually this is fantastic.’ Well, if you can’t block for it, if you can’t get your feet set up, you can’t get your quarterbacks’ eyes in the right place, you can put that thing in the trash. He understood those facets of the game as well as anybody I’ve ever been around. I’m just telling you, he’s as good as there is.”
As Manning’s mechanics and reactions became more refined under Caldwell, so did his judgment. Caldwell had played defensive back at the University of Iowa, and Dungy had been a safety for the Pittsburgh Steelers. Both men knew that defenders were preying on Manning’s habit of forcing a deeper ball when something less was available. Defenders counted on his impatience, they explained, and knew where he was likely to go with the ball.
“They’re hoping you’re going to make that throw,” Dungy told him. Manning should take that “as a challenge or even an insult,” Manning recalls the coaches telling him.
The alternative was to make enough short, safe completions that it would cause the defense to lose their patience. Pick away at them, lull them, fool them. And when they overplayed and tried to blitz? Blam! “Now you can go burn ’em with all the things you been doing for a long time,” Manning says.
As Manning mastered himself to watch and wait for his openings rather than force the issue, the teasing, baiting, exploitative style that would make him a Hall of Famer was fully born. He had always been good for 26 or 27 touchdowns per season, even as a rookie, but the problem was he threw nearly as many interceptions. In 2004, Manning exploded for 49 touchdowns — while throwing just 10 interceptions. Now he could toy with his intentions — and thus defenses. He could undersell or oversell them, force them to continually guess where he was going. He could fool and fleece a defense just by flicking his eyes deeper downfield.
For the next four seasons, Manning never threw more than 10 interceptions. Meanwhile, he lit the scoreboard at a record pace. In 2006, the Colts’ Super Bowl season, he threw for 31 scores while being picked off a career-low nine times.
“I felt like my game really took another step once [Dungy and Caldwell] got there, because of that insight,” Manning says. He would go on to throw for at least 30 touchdowns in seven of the next 10 seasons, including 33 in their 14-2 season in 2009 when the Colts returned to the Super Bowl with Caldwell as their head coach following Dungy’s retirement, and hit his peak of 55 in 2013 with Denver. Never again would he approach 20 interceptions in a season, even when neck injuries plagued him and eventually ended both his and Caldwell’s tenures with the Colts in 2011.
Life after Peyton
Caldwell went 24-8 in his first two seasons as a head coach, and when he suddenly crashed to 2-14 in this third year, there were “justifiable reasons,” as Flores’s lawsuit observes. Manning’s absence for multiple neck surgeries and his uncertain prognosis destabilized the entire team. The organization decided on a wholesale reset and committed to drafting Andrew Luck, leading to the ouster of both Manning and Caldwell. Despite what he had done with Manning, Caldwell wasn’t given the chance to guide the early stages of Luck’s career.
But he moved on with purpose, not a grudge. “I never felt that, ‘Hey, you guys did me wrong,’ ” Caldwell says.
What Caldwell did in the years that followed confirmed he wasn’t just a product of Peyton.
Caldwell never paid much mind to the terminology that announcers often used to credit the coaches most responsible for a quarterback’s success. Phrases such as “quarterback whisperer” or “offensive guru” were rarely assigned to him, but he didn’t mind. He never considered those terms compliments.
And the coaches who do catch those labels? “Perhaps,” he says, “when you look at their records, they probably haven’t done as much, and they probably only had one guy. One great one. Most of them only have one.”
Caldwell had another guy, right away. The Baltimore Ravens hired him, convinced he could help Joe Flacco lift a trophy. They had come within a dropped pass in the end zone of reaching the Super Bowl the previous season but felt Flacco could use a quarterback coach to take him to the next level.
In Flacco, Caldwell saw someone with the entire package: He had height. He could throw the ball far and hard, keep it on a line, with arch and touch. And he was unflappable. Caldwell told Ravens Coach John Harbaugh before taking the job, “If I can’t get this guy to play championship football, I need to be selling shoes or something else.”
Before the 2012 season, Flacco bet on himself, rejecting the Ravens’ offers for an extension and electing to play out his rookie deal in hopes of cashing in on greater riches. Caldwell endeared himself to Flacco early on, when he told him, “My goal and aim is No. 1, to win a title, but No. 2, to get you every dime of that money that you think you’re worth. We’re going to get that done.”
Studying film of every throw Flacco had made in his career, Caldwell noticed a flaw: Flacco would hold the ball too long because he had the arm strength to quickly zip passes into tight spaces. Caldwell worked with him on leading receivers with “a runner’s ball,” which helped wide receivers Torrey Smith and Anquan Boldin collect more yards after the catch and extend drives.
When the Ravens had a late-season swoon, Harbaugh turned to Caldwell to get them right. In December he elevated Caldwell to offensive coordinator and play caller, replacing Cam Cameron, a move that would pay dividends in the postseason. With Caldwell calling a faster-tempo game, Flacco went on a Joe Montana-esque run, throwing for 11 touchdowns and no interceptions in four games. They would slay the wunderkind rookie Luck and Caldwell’s former team, the Colts; Manning and the Denver Broncos; and Tom Brady’s Patriots, all on the road. Then they beat the San Francisco 49ers in the Super Bowl, with Flacco winning MVP. Flacco then signed the largest deal of any player at the time, worth more than $120 million.
“He just brought a sense of confidence that it was all going to come to life,” Flacco says of Caldwell. “It was just the simplicity of it and the matter-of-factness to it, when he would talk to you. ‘We’re going to do this.’ I think that we all kind of bought in and believed it.”
Flacco spent two seasons with Caldwell and never had to worry about which version of Caldwell would show up to the film room each day. Caldwell was considerate of Flacco’s opinions and preferences and consistent with his even-keeled temperament. “Those are the things that drive the ship a little bit, when you have a good team like we did,” Flacco said. “Because guys don’t want to come in and have to deal with somebody new every single day.”
That steady hand was what Stafford needed when Caldwell got his second NFL head-coaching job, with the Lions, in 2014. Stafford sat in on the interview process and was blown away by Caldwell’s knowledge of the game. “I was really, really impressed and really happy that they ended up hiring him and he became the guy,” Stafford said
The Lions were coming off two straight losing seasons. Caldwell wasn’t as hands-on with Stafford as he was with Manning and Flacco, but he worked closely with his quarterbacks coach and offensive coordinator Jim Bob Cooter on devising a plan to help the former No. 1 overall pick maximize his talent and put the team on his back. Stafford was a prolific five-year veteran, “an ultra-talented guy, really could do it all,” Caldwell says, but he was also very quiet. Caldwell challenged Stafford to be a more direct and demonstrative leader and hold his receivers to account if they weren’t running the right routes. “It needs to come from you,” Caldwell recalls telling Stafford. “Coaches can rant and rave, but when it comes from you, it’s different. Because they know, if they want the ball, they need to be where you tell them to be.”
He also again shored up a quarterback’s fundamentals with his technician’s eye. Stafford recalls a 52-38 loss against the New Orleans Saints in October 2017, when he had three interceptions and 10 of his passes were batted or tipped. Stafford didn’t tower over the line the way Manning or Flacco did, and he delivered the ball from a lower position.
Caldwell showed up to the next practice with a contraption that featured rods on wheels called “the baby,” which forced Stafford to put more elevation on his passes and clear tighter holes. Caldwell borrowed the idea from famed New York Jets coach Weeb Ewbank, who built a similar device to assist Joe Namath. But Stafford credits Caldwell with bringing it to a modern-era practice.
“He was always trying to, if there’s a problem, fix it without going crazy and overcorrecting a bunch of stuff,” Stafford says.
Under Caldwell, Stafford made his lone Pro Bowl appearance. In 2016, Stafford set an NFL record with eight fourth-quarter comebacks. The previous record of seven was set in 2009 — by Manning, under Caldwell.
“He’s as steady a human being as I’ve ever been around,” Stafford says. “Every interaction you had with him was really genuine. There’s no agenda, no anything, other than to genuinely get to know you and trying to understand you as a person or player. That makes it easy to get the connection. That trust.”
The Lions fired Caldwell after he went 36-28, concluding with consecutive nine-win seasons despite losing Pro Bowlers Ndamukong Suh in free agency and Hall of Famer Calvin Johnson to an early retirement. “Nine wins are not nearly enough,” GM Bob Quinn said.
Nearly every week since, the Lions have made the case that Caldwell’s firing was a mistake. The franchise has won just 22 games with two different full-time coaches in the past four-plus seasons. “I’m not sure, but it seems like Detroit might still be missing him after all these years,” Manning says.
Since his firing in Detroit, Caldwell has had just one more stint in the NFL, serving for a year with Flores as assistant head coach and quarterbacks coach in Miami in 2019. He interviewed for the jobs in Jacksonville and Chicago after last season and has been offered a handful of positions as coordinator, consultant or quarterbacks coach. But he’s holding out for another head coaching title, even as he has to compete against ever younger candidates. Though he is past corporate retirement age, he’s still younger than Bill Belichick and Pete Carroll. That he remains unemployed is baffling to former players. Saturday, who, ironically, was recently plucked from the broadcasting booth to become the Colts’ interim coach despite no serious sideline experience, says, “How he is not a head coach, I don’t understand.”
Until he gets the offer he believes he has earned, Caldwell will be content spending his early mornings staying fit on his Peloton and his non-football-watching evenings having fun by heading across the street with his wife, Cheryl, to play cards with neighbors Charles and Robin Paul, parents of NBA superstar point guard Chris Paul. The quarterback whisperer might need a bullhorn to get another opportunity, but he won’t ask for one.
“I feel that what’s due will come,” Caldwell said.
Emily Giambalvo and Clara Ence Morse contributed reporting.