Almost 40 million barrels of low-cost, price-controlled oil vanished last year in the government's complicated oil regulatory machinery, and nearly all of it later emerged selling for twice the price.

The vanishing oil, which continues to bewilder Federal Energy Administration officials, resulted in an illegal quarter-billion-dollar boost in consumer prices.

The oil, which "disappeared" on paper, came largely from wells in production before 1972, and should have been sold at the so-called "old oil' price of about $5.25. Instead, it was sold for the "new oil" price for over $11. New oil is a pricing category primarily reflecting output from wells in production since 1973.

Old oil centers FEA's reporting sytem shortly after it flows from the well. But the amount mysteriously shrinks between that first tabulation and the second, taken when it reaches the refinery.

"There is a human tendency here," says FEA Administrator John F. O'Leary, explaining how the oil vanished. "If oil sells for about $5 from one well, and over $11 from another, this is bound to happen."

Differences in the amounts of old oil in the system, which are subject to monthly reports, alerted FEA officials to the problem.

While no public action has yet been taken, The Washington Post has learned that FEA has issued a notice of probable violation to the Phillips Petroleum Co. for certifying old oil as new. Sources say the FEA notice, not yet made public, could result in requiring Phillips to refund $14 million for overcharges.

Other such notices are expected to follow.

According to FEA records, old oil vanished at a rate of about 108,000 barrels a day in 1976. This is 1.4 per cent of domestic production. Of that amount, 91,000 barrels a day "changed colors," as one baffled FEA officials said, to new oil selling at twice the price.

The other 17,000 barrels a day disappeared from FEA's reporting system altogether.

O'Leary and other FEA officials suspect that this oil was burned without refining by utilities and ships, or by the oil companies themselves to generate steam used to increase oil production in older fields.

FEA says some may have also been lost during swaps of crude, often old and new mixed, that industry makes to lower transportation costs.

The vanishing oil phenomenon which some FEA analysts initially thought might have been a statistical aberation, is now recognized as a serious problem. The agency is stepping up its auditing, especially of oil brokers who would be in the best position to garner the $7 profit boost as oil changes colors.

In March and April of this year the amount of vanishing oil climbed to about 2.5 per cent of domestic production.

"We are very concerned about this, although we are still unsure this is a trend or the result of inventory changes, says Robert B. Nordhaus, recently appointed assistant FEA administrator for regulartory programs.

In the next weeks FEA will publish a series of proposed regulations, targeted at vanishing oil, to tighten up industry's compliance with the agency's over 20,000 pages of regulations.

It is more likely to be the smaller companies than the larger ones," says one FEA official.

While FEA analysts have been aware of the vanishing oil problem for some time, it only recently commanded attention at the administrator's level. In part this was due to the prodding of a Mobil Oil Corp. vice president, Bonner H. Templeton, who wrote O'Leary April 15 saying the end result of vanishing oil "is an apparent windfall to a few individuals firms, at the expense of others."

Companies like Mobil, which have large amounts of old oil, are required under FEA regulations to pay money to companies that have mostly new oil. This buying of "entitlements" is designed to offset the advantage from having available more of the cheaper old oil.

Thus, if a company is able to sell old oil as new, it can collect not only that additional profit but also entitlements from other companies.

Another executive concerned about vanishing oil is Marty Weiland of Standard Oil Co. of Indiana (Amoco), who says vanishing oil "may be an outright fraud . . . it means higher costs for us, and higher prices for the consumers."

"The guy who probably benefits from this is the reseller, rather than the producer," says Weiland.

Amoco, like Mobile, is a major entitlements purchaser; last month it paid $40 million.

The vanishing oil was discovered during FEA efforts to track old oil through the complicated reporting system.

Curiously, producers do not report to FEA how much oil they produce. Instead, oil enters the FEA numbers system each month on the "Domestic Crude Oil Purchaser Report" originated by the 1976 Energy Policy and Conservation Act. This monthly report includes a listing of the amounts of different priced oil, old, new, and other counted as "purchased."

The next tracking point in FEA's reporting system is the refinery, through the monthly "Domestic Crude Oil Enthtlements Program Refinery Report."

Old oil vanishes or changes color to higher-priced oil between these two reports, FEA officials say.

The problem, some administration officials admit in private, is a result of the complexity of the FEA regulatory program, which burdens the industry with bales of forms to meet reporting requirements set forth by Congress.

A recent study prepared by FEA and the Council of Economic Advisers concluded that FEA's regulations are unenforceable to a large extent, unsuitable for shortage periods, and would cost the industry $570 million a year to reach 100 per cent compliance.