Simon Cowell told the Hollywood Reporter back in August that an audience of less than 20 million for the launch of his “X Factor” would be a flat-out “disappointment.”

Turns out, “X Factor” — which Simon wants to displace his old program, “American Idol,” as the biggest show on TV — attracted 11.9 million viewers Wednesday night as it began its second week on the air. And a week earlier, Simon’s new singing-competition series averaged 12.5 million viewers.

Instead of the whole country talking about “The X Factor” — as they do in England, as Simon has assured us — the show has been eclipsed by this new TV season’s Big Story: “comedy.”

ABC’s Emmy-winning “Modern Family” once again beat “The X Factor” on Wednesday night, attracting not only a bigger crowd — 13.5 million viewers — but also outstripping the singing competition among the younger, 18- to 49-year-old viewers both shows target.

“Modern Family” also enjoyed its best-ever ratings among women between the ages of 18 and 34. That group is supposed to be “The X Factor’s” sweet spot.

As further evidence that “The X Factor” has so not sucked up all the available air on Wednesday nights: ABC managed — in the teeth of Simon’s new show — to bag its biggest comedy premiere at 8:30 p.m. in eight years. The unveiling of its sitcom “Suburgatory” — about a teenage girl dragged from New York City to the suburbs by her single dad — attracted 10 million viewers, or about 1 million more people than its lead-in, “The Middle.”

That is a major accomplishment opposite the singing competition that was supposed to be this fall’s phenomenon.

In the days leading up to “The X Factor” launch, Simon, the former “Idol” producer/judge, said he was not competing “for the silver or the bronze.” On Wednesday, his “X Factor” did just that, snaring the night’s third-biggest audience, behind ABC’s comedy series and the silver-winning CBS drama “Criminal Minds” (12.6 million viewers).

But Simon’s new super-slick singing series — in which he and celebrity judges Paula Abdul, Nicole Scherzinger and L.A. Reid choose and mentor a crop of pop-singing hopefuls for us to vote on — has done a terrific job of getting Fox into the game during the fourth quarter, which has been a challenge for the network for years.

Thanks to an all-“X Factor” lineup, Fox did wind up with prime-time’s largest audience for the night, and the best rating among English-language broadcasters among the 18- to 49-year-olds.

It’s just not the phenomenon that Simon assured us to expect.