The recent jump in energy prices has been portrayed as an act of retribution by fossil fuels after years of low investment. The International Energy Agency’s latest long-term outlook trumps that: We’re underinvesting in energy, period.

Rather than some neglected soul seeking vengeance, the global energy system might better be thought of as a particularly confused, and volatile, tween. It’s a phase we’re all going through — just a phase that happens to be of indeterminate length, course and outcome — as we change over from a mostly fossil-fueled society to a decarbonized one.

The sun-powered uplands of 2050 envisaged in the IEA’s net-zero emissions scenario are cleaner and mitigate the ravages of climate change. The issue is getting there. In particular, relying on two energy systems — one carbon-based, the other not — that are broadly opposed but also intertwined in certain respects is a recipe for mismatches in supply and demand.

Contrary to the simplistic narrative of green activists crowding out new wells and mines, the current tightness in energy markets results from multiple issues colliding at once. These include the recovery from Covid-19, an investment strike in U.S. shale that has more to do with “G” than “E” or “S”, OPEC+ taking the opportunity to hold out for higher prices and a cyclical lull in expanding liquefied natural gas capacity.

There is no doubt, though, that the growing imperative to decarbonize will have a bigger impact as time goes on. The IEA’s own net-zero roadmap, released in March, caused a stir by positing a future with no investment in new fossil-fuel supply.

That would be fine if that fossil-fuel line bending down was mirrored perfectly by a zero-carbon energy line rising to eclipse it. The IEA models that as a scenario but does not see evidence of it happening now:

However, clean energy investment still remains far short of what is required to put the energy system on a sustainable track. At the same time, the amount being spent on oil and natural gas is also short of what would be required to maintain current consumption trends. A surge in clean energy spending is the obvious way out of this impasse, but something has to change quickly or global energy markets face a turbulent period ahead.

Endless headlines about recovery packages may suggest a river of public money is flowing into renewables. Yet the IEA says that, of roughly $16 trillion governments threw into the pandemic’s economic breach, less than $400 billion, or 2.4%, was directed at clean energy infrastructure. The IEA’s estimated investment requirement to reach net-zero emissions by 2050 is, for this decade, $4 trillion — per year. That’s four times the average spent over the past five years.

In the meantime, however, we’re still using a lot of oil and gas. Even on the net-zero pathway, the latest World Energy Outlook puts oil demand in 2030 at about 65 million barrels a day. Natural gas, meanwhile, plays a near-term role in furthering transition by providing backup to intermittent renewable energy until storage options such as batteries proliferate; demand is only 18% lower by 2030.

Yet an oil and gas industry geared to growth will, like any other industry, focus more on what’s lost than what’s left, and capital costs will adjust accordingly — way up. The fossil-fuel supply cycle is inherently long, especially when relatively nimble shale is subdued, making it prone to periodic shortages and excess. The transition supercharges this by deterring investors remembering past sins and worried about future stranded assets.

Further volatility comes courtesy of geography. As demand for oil plateaus and declines, supply retreats to its petrostate heartlands; the IEA has Russia and the OPEC members supplying 61% of oil in a net-zero 2050 versus 47% today. Of course, the pie is way smaller, so the absolute number of barrels they’re selling drops by 70%. Some may end up like the United Arab Emirates, but we can also expect more Venezuelas.

We are, therefore, likely to face more episodes of tight or volatile energy markets. Reverting back to fossil fuels isn’t an option. That the IEA has released at least two tomes with a heavy emphasis on net-zero emissions this year sends a strong signal about where its member governments’ heads are. Spurring faster investment in clean energy technology, especially demand-side elements such as electrification and efficiency, represents the climate-friendliest path to mitigating volatility.

That big $4 trillion number should also be set in the context of savings; this figure is, after all, investment rather than just running costs. For example, at $50 in real terms, the annual saving from reducing oil demand alone would be about $1.2 trillion by 2050 and more than $1.7 trillion if you kick in $50 a tonne for avoided carbon emissions. Moreover, the switch from an energy system geared less to commodity flows and more to infrastructure availability should provide more of a cushion against traditional price shocks.

Yet just focusing on raising renewables isn’t enough in the immediate term. Subject consumers to enough anxiety about this most vital service, and they have a way of forcing changes of their own, either via recession or, more relevant for renewables, a political backlash. Managing this means altering the very foundations of society’s energy investment choices.

Currently, the world subsidizes clean-tech development while paying much more for a reasonably steady flow of fossil fuels. We’re entering a world where we must adequately price the benefits of renewables to ultimately render subsidies unnecessary. But we must also find ways to value some fossil fuels, for some time, and increasingly for their capabilities as backup rather than continuous supply. A good example, and one we saw this summer in California, is shifting gas-fired power plants from all-day baseload to meeting peak demand.

We do need to throw money at the problem. With a system this prone to spasms, though, our throw must be exceptionally well-aimed.

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Liam Denning is a Bloomberg Opinion columnist covering energy, mining and commodities. He previously was editor of the Wall Street Journal’s Heard on the Street column and wrote for the Financial Times’ Lex column. He was also an investment banker.

More stories like this are available on bloomberg.com/opinion

©2021 Bloomberg L.P.