Ford Motor Co. Chairman Philip Caldwell said today that the budget impasse between Congress and the Reagan administration is undermining hopes for the recovery of the auto industry and the entire economy.

"The whole country is dismayed and increasingly angered over the failure of the executive and legislative branches and the Federal Reserve Board to agree on steps to remedy the federal budget and the interest-rate problem," Caldwell told 1,100 people attending Ford's 27th annual stockholders' meeting here.

Caldwell said that the automaker's second quarter will be "substantially better" than the first three months of the year, when Ford lost $355 million. But further progress will depend on the economy, he added. He also declined to say when Ford would resume dividend payments to shareholders.

"High interest rates represent the most distressing element of our economic turmoil. They are a persistent economic and psychological deterrent to plans for buying new homes and cars," Caldwell said.

"We need to get the budget problem solved and the interest rates down, and we need to do it in the near term," the Ford chairman told reporters after his address. "When the cost of money comes down, Ford Motor Co. will be in a better position to win on every front than at any time in its history," Caldwell said.

Ford last year lost $1.06 billion in worldwide business, most of it in North America. The company skipped dividend payments in the first quarter this year and plans to omit the dividend in the second quarter, facts that weighed heavily on stockholders at today's meeting.

Caldwell said "it was with great reluctance" that Ford's board of directors decided to omit the dividends. But he said "the plain fact is . . . that any other decision would have been irresponsible." The continuing depression in the North American car and truck industry, combined with an $11 billion investment in new product development, left Ford "no alternative," he said.

Auto industry analysts generally agree that high interest rates are a major psychological impediment to new-car purchases. But some, including L. R. Windecker of Ford, say that general uneasiness about the economy is an important factor in disappointing auto sales.

"Interest rates alone aren't doing it," Windecker said in an interview following the stockholders' meeting.

"The basic problem is that people are uncertain about the economy. The guy looking over his shoulder wants to be sure that the man who was just laid off behind him will be the last man to lose his job," Windecker said.