Businesses plan to cut back investment this year despite new tax incentives that President Reagan said would boost spending on plant and equipment, the Commerce Department reported yesterday. At the same time, the department reported poor Christmas sales for the nation's retail stores.

Firms surveyed in November and December of last year told officials that they planned to spend $346.4 billion on new investment this year, equivalent to a real drop after inflation of 0.5 percent from last year. In 1981 there was a real increase in business investment of just 0.3 percent, the survey published by the Commerce Department showed.

Meanwhile, the department reported that retail sales rose by only 0.4 percent last month after seasonal adjustment. In December 1980 sales rose by 0.8 percent on the same measure. The recession has thus taken its toll on retailers. Deputy Commerce Secretary Joseph Wright Jr. said yesterday "retail sales are essentially unchanged since July and only 4.9 percent higher than a year ago."

Inflation in the 12 months to December was more than 5 percent, so there was been a fall in the actual volume of retail sales over the year. Stores have generally succeeded in keeping up volume only by cutting prices and putting goods on special promotions.

These helped to produce a "late surge in Christmas sales of household durables and general merchandise," Wright said. But retailers "may remain cautious in re-ordering" as they have not been able to make much if any profit on the promotions. On top of the recession "the severe weater across the country is holding down store traffic," Wright added.

The recession has also hit business plans for investment. Last year Reagan predicted that his economic program, including large retroactive tax cuts for business, would lead to sharp rises in investment. However, the economic slowdown is "having an impact on capital spending plans" Commerce Department economist Robert Ortner admitted said yesterday.

"Weak profits, low capacity, declining production" and great concern that sales may continue to be weak are naturally making businessmen scale back their investment plans, Assistant Commerce Secretary for Economic Affairs Robert Dederick said yesterday.

However, both he and Ortner claimed that the tax cuts now in place will encourage a stronger pick up in capital spending during the recovery. They will "put a floor" under the present downswing, Dederick said.

Albert Wojnilower, economist for the First Boston Group, yesterday predicted the current recession probably would end within the next two or three menths but said long term interest rates would continue at record levels without any significant economic expansion.

Nonmanufacturing industries plan a 1.4 percent decline in spending this year, offsetting an increase of 0.9 percent for manufacturers. The auto industry forecast "flat" spending in dollar terms this year.

Electrical machinery manufacturers anticipate a "pretty healthy" increase of 20 percent in dollar spending, Ortner said. However, declines are forecast by aluminum and copper producers, glass stone and clay manufacturers and paper companies.

Initial Commerce Department estimates for increases in business spending in 1981 proved to be too optimistic. But in times of recession such as at the end of last year when the latest survey was carried out, the forecasts are likely to prove too pessimistic, Dederick said.

Meanwhile, the Ethyl Corp. announced yesterday that it was postponing a multimillion-dollar expansion project planned for downtown Richmond because of high interest rates and the high cost of transferring employes from St. Louis to Richmond.