Prices have been rising across many areas, but the one we’re likely to notice most this winter are energy bills. This won’t just affect household budgets; there will be political repercussions too.

Dramatic increases in U.K. and European wholesale power prices in recent weeks are driving up the cost of energy for consumers, with expectations for bigger hikes during the winter months when demand soars. The British media has already begun warning of a rise in “fuel poverty.” Addressing this won’t make the government’s next big task any easier: The U.K. is about to host the United Nations’ COP26 climate summit in a couple of months and will need to spell out how Britain will pay for the transition to net zero.

Balancing the imperatives of affordability and sustainability has never been easy, but the timing of this collision looks particularly awful.

The temptation will be to heap more costs on the small and medium-sized businesses that supply energy to consumers. But not only are these companies needed for innovation and investment in the sector, the pathway to net zero runs directly through their supply lines. 

The consumer focus stems both from the politics of energy and the history of deregulation in Britain. U.K. energy regulator Ofgem was set up in the late 1990s to protect consumers from price-gouging by what was then the “Big Six” suppliers. In its annual survey of consumer attitudes published last week, the report card was glowing: Nearly three-quarters said they were satisfied with the ease of understanding their bill and its accuracy. Just as many reported being happy with the customer service they are receiving.

Take a survey of energy retailers, however, and things don’t look so rosy. More than two dozen retailers have left the market since 2019. And the pace may be picking up: Last week two suppliers – Pfp Energy and Moneyplus Energy – announced they were going out of business. Those in the sector say more will leave the field in the months ahead.

Some of the exits were unsurprising — weaker players lacked the right hedging strategy, a large enough customer base or were underfunded. But it’s not quite right to say this is just Schumpterian capitalism at work. In an environment of volatile prices, many suppliers are facing negative margins in part because Britain’s price cap on energy and a host of other regulations mean the costs of provision are above what they can charge consumers. 

It’s hard to imagine more things going haywire at once. The price of gas, carbon and coal — the fossil fuels components that set prices — have all risen sharply. Wind output has dropped, while Russian gas flowing into Europe has been lower than expected. Weather extremes have meant higher demand for both heating and cooling in Asia and Europe. There have been outages in nuclear and other facilities. The U.K. had also reduced its gas storage capacity, making it more vulnerable. Prices have risen all over Europe, too.

Even Brexit costs factor in here as the trading of energy through interconnectors — the physical network connecting Britain to the Continent — is more costly to manage. Carbon prices rose by almost 50% between the April and September, while gas prices have risen by almost 200% in that time.

Especially if it’s a cold winter, prices are only going to rise further, and there are concerns about supply too. The U.K. relies heavily on gas for heating and electricity production and imports around half its gas supply from Europe. Energy costs rose by around 80% to 747 pounds ($1,031) on an average consumer bill in Aug. 2021, from 410 pounds in 2020. Last week Ireland had to halt wind power exports to the U.K. and has warned of potential blackouts.

In a normal market, prices would just rise to reflect higher input costs. Indeed they are, but not enough to cover supplier costs. That’s because under Theresa May’s government, the Conservatives borrowed a Labour idea and backed a legally mandated price cap. Twice a year Ofgem sets the maximum price that can be charged on a standard variable rate. In August, it raised the price cap that applies from next month. But the cap increase is less than half the rise in energy costs suppliers are facing, so many companies are still forced to make a loss.

When an energy retailer fails, their customers (94,000 from the two exits last week) get redistributed to other suppliers. Going under is part of business, but serial failures begins to look careless. There are now just under 50 retail suppliers, from over 70 a few years ago.

Larger legacy suppliers such as EDF Energy, especially those with their own trading arms or even generation capacity, are better placed to weather the storm. Some others — such as Octopus Energy — are well capitalized and focused on accumulating bigger customer bases. Unsurprisingly, neither of these groups are making a fuss about the price cap despite seeing margins eroded.

Smaller suppliers, however, point out a threat to their viability and to innovation and investment for cleaner energy. “In no other sector is the government forcing a loss on the supplier,” says Steven Day, co-founder of Pure Planet, a supplier of green energy serving 250,000 homes and 24% owned by BP PLC. “The cap is fixed like some kind of tablet in stone from God; the market doesn’t work like that. Whether it’s quarterly or monthly; it needs to be calculated more frequently.”

But while a price-cap review twice a year now looks too seldom, changing it is not so straightforward. 

Dermot Nolan, who was Ofgem’s chief executive officer for six years until 2020, told me they originally considered putting in a price cap every two or three months and decided against it. “There’s certainly a case for moving to every four or two months now,” says Nolan, currently a director at Fingleton, which provides strategic regulatory advice. “It would probably avoid distortions, but the question is would consumers tolerate prices changing every month.”

And yet, something will have to change. The cap, as Nolan also acknowledges, is simply not designed for the emerging energy landscape that will require a transformation in home heating, a lot more electric vehicles, charging points, solar panels and other green technologies in the home.

When that discussion will happen is not clear. Robert Buckley, a consultant with energy specialist Cornwall Insight, notes that the government may not have appreciated how its actions would impact the supply side of the market when it decided to extend the price cap, but the political reality is that it’s unlikely to change that now.

Meanwhile, a thousand small regulatory cuts are probably hurting just as much as the price cap. Strict controls on billing, tariff structures, the installation and rental costs of smart meters and the payments of so-called renewable obligations are well-intended but often add costs while making it harder to innovate and making the sector unattractive for investors. 

A government that says it wants less red tape should be encouraging a lighter touch, or at least a smarter one. If it wants a market-based energy sector (not the nationalization the Labour Party backed) it should also be driving a real discussion of how the transition costs to net zero will be shared and what profits are reasonable. Regulation is essential in energy provision; but it shouldn’t decide the winners and losers.

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Therese Raphael is a columnist for Bloomberg Opinion. She was editorial page editor of the Wall Street Journal Europe.

More stories like this are available on bloomberg.com/opinion

©2021 Bloomberg L.P.