In the world of executive compensation, one idea has long been considered gospel. Chief executive pay, companies say, is tied to the returns they produce for shareholders, and the "pay for performance" concept is used to defend the lofty stock-based compensation that makes up the majority of most large public companies' pay packages.

But a new study released Thursday by the investment research firm MSCI, reported earlier in the Wall Street Journal, raises questions that could seem like sacrilege in the CEO pay world. It found that 61 percent of the large public companies it studied had 10-year shareholder returns that were, to use the name of the report, “out of whack” with the pay CEOs took home over the same period.

From 2006 through 2015, 23 of the 423 companies in the study had underpaid CEOs who delivered high performance, the report says, while 18 companies overpaid their CEOs for below-average returns. Only 163 of the companies, or about a third, had shareholder returns that were “generally well aligned” with CEO pay. The report summarized this way: "These findings suggest that the 40-year-old approach of using equity compensation to align the interests of CEOs with shareholders may be broken."

Ric Marshall, executive director of the research team at MSCI that focuses on environmental, social and governance issues, said in an interview that the 10-year time period is critical for studying whether CEO pay is really tied well to company performance. “Over the short term they may be well-aligned, but over the long term it’s so evenly distributed that it’s almost a random sampling,” he said, pointing to a chart in the report that shows a scattershot relationship between total shareholder return and cumulative take-home pay over a 10-year period. “It looks a lot like you threw paint on the wall.” 

The study examined what executive compensation experts call “realized pay,” or what CEOs actually pocket each year after exercising stock options or becoming eligible for stock grants, a figure that is often quite different from the annual “awarded pay,” or the value of an executive's compensation when it is granted. Last year, Marshall examined the relationship between those figures and shareholder returns, and found a similar misalignment.

The study's finding was not surprising to some advocates for improved CEO pay practices. “This lines up exactly with what we’ve been saying,” said Roseanna Landis Weaver, an executive compensation expert at the nonprofit As You Sow. “Pay for performance is much more of a mantra than a statement of fact.” (Weaver notes she formerly worked for a subsidiary owned by MSCI, but has no financial ties to the company.)

Others were less convinced. David Wise, a senior client partner at Korn Ferry Hay Group who advises companies on executive pay, questioned how the study accounted for the size of companies — CEOs of larger, more complex companies are typically paid more than leaders of smaller ones — as well as how “realized pay” deals with executives' individual timing for exercising their options. “It looks for a relationship between an apple and an orange,” Wise said of the study.

Marshall said he tested the overall results by relatively sized companies and found no significant difference in the results. Equity sales or metrics may have had an impact on short-term results or a smaller sample size, he said, but they were an unlikely concern for a study with a 10-year time horizon involving more than 400 companies.

The report did find that CEO pay and shareholder returns were better matched over five-year or even three-year periods, the time frame used in many CEO pay packages. But as the nature of investing changes, with more money held in large-cap indices for longer periods of time, Marshall argues that CEO pay needs to be better tied to a longer time horizon. While the report does not name companies that overpay or underpay their CEOs — he says that wasn't its point — he did point to ExxonMobil, which vests executive shares over an unusually long period of time, as one company that is doing things differently.

Marshall's concern, he says, is not whether CEO is too high or too low, but whether it's in keeping with company performance. “Eighteen companies in the sample [have] CEOs who have performed below their peers and yet taken home, on average, $10 million per year over the entire 10 years,” he said. “To me, that's not right.”

Read also:

Like On Leadership? Follow us on Facebook and Twitter, and subscribe to our podcast on iTunes.