Soaring energy costs are pushing inflation up and the economy down.

Led by a 5 percent jump in the price of gasoline, consumer prices rose 1.1 percent in May, the Labor Department reported yesterday. It was the fourth month in a row with an increase of 1 percent or more.

In Washington, consumer prices were up less, rising 1.6 percent during the two months from March to April, compared with 2.2 percent nationally. (Story, Page A6.)

Meanwhile, the Commerce Department tentatively concluded that economic activity across the country declined at a 2.4 percent annual rate in the last three months, bringing the nation to the brink of a recession.

Whether the economy drops again in the third quarter probably depends on whether the 13.6 percent inflation rate of the last three months moderates, a top Carter administration official said. Any easing of inflation, in turn, depends on what happens to energy prices.

Energy costs to consumers went up 4.2 percent in May alone, and gasoline increases at the pump have been rising at a 55 percent annual rate so far this year, the Labor Department report said.

In unusually candid testimony before Congress' Joint Economic Committee, Barry Bosworth, director of the Council on Wage and Price Stability, said, "I see no significant moderation in consumer prices for several months."

Earlier, Bosworth and other administration economists had been hoping that an easing of food price increases, which began to show up in May, would mean smaller increases in the overall Consumer Price Index.

Now, said Bosworth, "Energy prices have replaced food as the major problem."

The cost of food at grocery stores rose 0.5 percent in May, the smallest monthly increase since August. However, beef and veal prices were up a sharp 3 percent. Both poultry and pork prices declined.

Most other grocery store prices went up more in May than they had in April, particularly for cereals and baked goods.

Bosworth expressed the fear that the recent rapid runup in grain prices - the result of expected poor crops in France, a major food exporter, and the Soviet Union, a large importer - could lead to unexpectedly higher food prices next year.

Agriculture Department officials, meeting with reporters, took a cautious view of the grain price situation. Economist J. Dawson Ahalt, chairman of the World Food and Agricultural Outlook and Situation Board, said he was less certain than he was several weeks ago that food prices would moderate in coming months.

Dale E. Hathaway, assistant secretary for international affairs, said it appeared that the Soviet Union has contracted to purchase 13.2 million metric tons of U.S. grain in the 12 month period ending Sept. 30, and is likely to buy 15 million tons, the maximum allowable under a U.S.-Soviet agreement.

Ahalt said that record large grain stocks are providing some cushion against still higher prices. A major question is the extent to which grain prices will be held down by the staged withdrawal of incentives to farmers to hold grain off the market in their own storage facilities. The incentives under the program are withdrawn as market prices rise.

In his JEC testimony, Bosworth recommended that the nation should "move as fast as possible to all-out production," and that there should be no set-asides next year for corn, wheat and other such crops. Set-asides are requirements that farmers keep a portion of their land idle in return for certain income supports.

The COWPS director, who has resigned effective the end of July, was asked if he was leaving out of a sense of frustration.

"Party," Bosworth replied.

The sort of frustration he feels became evident as the hearing focused on energy policy.

"We have to develop a national consensus" on what energy policy ought to be, he said. "We are not getting very far."

Nor is there any consensus within the administration, Bosworth indicated. "You can't get the government to the position that it has the same explanation two days running for a problem in an area," he said.

Policy is torn constantly between competing concerns over the price of energy and its availability. The result is, Bosworth said, that government changes its policies "every 10 days or so . . . It used to be every year."

The Commerce Department's estimate of a drop in GNP in the second quarter was based on very preliminary figures for some parts of the economy and estimates of others.

Sometimes this estimate "hits it, sometimes it's way off," one administration economist cautioned.

The drop was concentrated in consumer spending, much of which was the result of higher energy prices leaving less room for other buying and also hurting auto and truck sales.

If a recession comes, Bosworth warned that it would not stem the inflation tide. "It won't work," he declared. "The costs will get too high."