Former Obama White House senior adviser Valerie Jarrett, pictured in 2009, called out Forbes for including just one woman on its list of top innovators. “If your methodology produced only one woman out of the 100 most innovative leaders, obviously you should have challenged it rather than publishing it,” Jarrett tweeted. (Marvin Joseph/The Washington Post)

Forbes Magazine may be best known for its lists of the world’s wealthiest billionaires, but its recent ranking of the most innovative CEOs is the one that has been getting the most attention — and not for good reason.

Last week, Forbes published its list of 100 CEOs — or as it called them, “the most creative and successful business minds of today” — the top of which included a largely predictable catalogue of tech titans and billionaires including Tesla’s Elon Musk and Amazon founder Jeff Bezos (who tied for first; Bezos is the owner of The Washington Post), Facebook’s Mark Zuckerberg and Apple’s Tim Cook (No. 3 and No. 8, respectively), and Google’s Larry Page and Sergey Brin (tied for No. 10).

But readers had to scroll all the way down to No. 75 to find the first — and only — woman on the list, Ross Stores CEO Barbara Rentler. A photo of her was not included.

Social media users quickly took notice. “Come on @Forbes, WHAT WERE YOU THINKING???” tweeted Sarah Robb O’Hagan, the former CEO of indoor cycling studio Flywheel Sports and former president of Equinox gyms. “I can come up with 100 women at this level without even googling. FIGURE IT OUT.”

Author Anand Giridharadas noted that “there are twice as many men named Stanley as there are women of any name” on the list. “And there are only two Stanleys.”

And Nextdoor CEO Sarah Friar, who is on the board of directors of Slack and Walmart, tweeted: “At first I thought maybe they had a men’s and women’s list,” followed by the slapping-my-head emoji.

By Sunday, Forbes’s editor, Randall Lane, had posted a response to the uproar over the list, admitting the methodology was “flawed."

On Monday, as the outcry continued, Lane said in an email to employees that he was announcing a task force that would study how it “somehow missed the forest from the trees” and recommend how to design lists and research projects more fairly, saying, “We deserved the backlash.”

In a tweet Monday afternoon, he wrote: “We blew it. Now we’re doing what journalists do: figuring out how this happened and learning from it.”

In his Sunday post, Lane wrote that the list wasn’t subjectively decided by a group of editors sitting in a room but was based on a methodology it had been working on for years with professors at Brigham Young University and INSEAD.

It ranked CEOs with greater than $10 billion in market value by their companies’ “innovation premium,” which the list’s methodology described as the “difference between their market capitalization (value) and the net present value of cash flows from existing businesses,” as well as things like the track record of the CEOs’ performance, their social capital and their reputation in the media for innovation.

But starting with just CEOs of the largest publicly traded companies created a problem from the beginning. The “pool ultimately proved the problem,” Lane wrote, as women represent just 5 percent of CEOs in the S&P 500 index. “In other words, for all our carefully calibrated methodology, women never had much of a chance here.”

The methodology was flawed, Lane wrote. “While each data point individually made logical sense, as did focusing on data-rich public companies, the entire exercise collapses if the possible ranking pool doesn’t correlate at least somewhat with the overall pool of innovative talent. It would be intellectually dishonest to construct a methodology designed to generate a predetermined result, but in this case the forest got lost in the trees.”

Lane’s admission was one many women responding on Twitter had already noted. Such rankings may appear to be data-driven, but if the sample is heavily populated by men, or the methodology relies on inputs that could be biased, the results of the exercise may not be objective.

For instance, the angel investing network #Angels, founded by female tech executives and investors, called the list “embarrassing,” saying “the methodology — limiting ‘leaders’ to US CEOs of $10bn market cap cos, weighting media perception and social following — just compounds existing biases. Do better, Forbes.”

Herminia Ibarra, a professor of organizational behavior at London Business School, tweeted before Lane’s post Sunday, “What’s astounding is that there is no discussion of how much the 99% male list might be a result of choices about what quantitative indicators to use.” Others laced their comments with sarcasm: “It’s so weird,” mocked one Twitter user. “Our algorithm, which is based on years of outdated statistics that favour privilege & patriarchal systems ... just seems to keep rewarding white guys.”

Given the results, some wondered whether the article should have been released at all. (In his post, Lane said Forbes should have used this situation, as it has on past lists, “to delve into the larger problem of women ascending to CEO. We own that.”)

Valerie Jarrett, a former senior adviser to President Barack Obama, lodged this complaint on Twitter: “Come on, @Forbes. If your methodology produced only one woman out of the 100 most innovative leaders, obviously you should have challenged it rather than publishing it.”

Read also:

How having more than one woman vying for a job can change everything

Like On Leadership? Follow us on Facebook and Twitter, and subscribe to our podcast on iTunes.