The economic recovery got off to a more modest beginning last quarter than many economists had expected, with the gross national product rising at an annual rate of only 3.1 percent after adjustment for inflation, the Commerce Department reported yesterday.

Real GNP fell at a 1.1 percent rate in the fourth quarter of 1982 as the longest recession in post-war history came to an end. On the basis of much less complete information than is available now, Commerce estimated last month that first quarter GNP was rising at a 4 percent rate.

Even with the relatively small first quarter gain in GNP--the total output of goods and services for the American economy--Robert Ortner, chief economist at Commerce, said that "it's still very much up in the air" whether the current quarter's increase will be as large.

Forecasters nevertheless remained confident the recovery would continue, perhaps unevenly but rapidly enough to lower the current 10.3 percent unemployment rate somewhat in coming months.

Ortner and other economists expect to see faster growth in the second half of the year, and yesterday Commerce Secretary Malcolm Baldrige said, "The economy is on track toward the administration's projection of 4.7 percent growth during the four quarters of 1983." That goal is still nearly two percentage points lower than the usual increase for the first year of a recovery.

Inflation, as measured by the fixed-weighted GNP price index, fell from a 4.9 percent rate in the fourth quarter to a 3.2 percent rate in the first, the lowest quarterly rate in 11 years. That index is not affected by relative changes in production of the various components that compose GNP.

On the other hand, the implicit price deflator--which gives weight to price changes of different goods and services according to their actual share of total output in each quarter--rose at a 5.8 percent clip, up from a 3.7 percent rate in the previous quarter.

One reason for the cautious outlook for the second quarter by Ortner and other economists is that final sales of goods and services went up at only a 0.9 percent rate in the first quarter, far less than the 5.4 percent rate of the fourth quarter. Spending by consumers was up at a 2.3 percent rate.

Baldrige said he is optimistic in part because he expects "to see stepped-up growth in personal consumption in response to higher employment, above-average income tax refunds, the July 10 percent personal income tax cut, and greater consumer confidence."

Meanwhile, much of the impetus behind higher production has been the desire of businesses to halt a decline in their stocks of materials and finished products waiting for delivery. The level of business inventories, which had fallen at a $48.3 billion annual rate in the fourth quarter, dropped in the first quarter at a much smaller $28.5 billion rate.

Thus, production in the quarter was still lower than the level of sales, and additional increases in output will be needed to bring the liquidation of inventories to a halt.

Economists are counting on that stimulus to production, along with strong increases in residential construction, to keep the recovery going in the short run even if consumers do not step up their spending. The increase in real outlays for housing construction was second only to the change in inventories in contributing to the overall first quarter gain in GNP.

Surprisingly, spending by business for new plants and equipment also rose at a 2.7 percent rate in the first quarter. With the use of existing production capacity so low, forecasters had expected such outlays to decline for another quarter or two before turning upward.

Meanwhile, both federal government spending and net exports fell in the first quarter. The drop in federal government spending was the result of some large purchases of grain and dairy products by the Commodity Credit Corporation in the fourth quarter that were not repeated in the first quarter.