The nation's financial markets swung widely yesterday amid rumors -- which dealers and a U.S. Treasury official said were unfounded -- that Arab governments were selling dollars and withdrawing their money from the United States.

The stock market took a pounding for the second day in a row, accompanied by a sharp fall in the dollar, but the markets recovered some of their losses by the end of trading.

The Dow Jones Industrial Average, which plunged a record 39 points Wednesday, was down 22 points at one time yesterday, but stocks rallied and the Dow finished with an 8.38-point loss to 1,518.23.

In Europe, the dollar fell sharply, especially against the West German mark, as the rumors multiplied that Arabs were reducing their U.S. investments in retaliation for President Reagan's call for economic sanctions against Libya. But bankers and dealers said they saw little evidence of Arab selling, and the dollar later stabilized.

The dollar was quoted in New York at 2.441 marks, off from 2.4375 Wednesday, and at 201.65 yen, down from 202.65. It cost $1.4570 to buy a British pound, compared with $1.4430 late Wednesday. Gold prices surged $10 an ounce in heavy trading as the rumors spread, but later settled back, closing in New York at $338.30, up $3.90.

Assistant Treasury Secretary for International Affairs David Mulford said that the Treasury had seen no signs of dollar-dumping by Arab governments, but added that general market nervousness over the freeze of Libyan assets, as well as worries in Europe over the sharp drop in U.S. stock prices, probably had motivated the slide in the dollar.

Mulford said that "when people over there in Europe got up in the morning yesterday , they may have said, 'It looks serious, maybe we ought to close some dollar accounts down.' "

But he said there was no evidence, "just rumors," of Arab nations switching out of the dollar. "If there were Arab government selling, we'd know about it," he said. Sources in the New York financial markets confirmed the Treasury report.

Mulford acknowledged that there probably was uneasiness caused by Wednesday's White House order freezing Libyan assets in this country, one of the responses to charges that Col. Muammar Qaddafi encourages Middle East terrorism. "Whenever assets are frozen, you can anticipate market nervousness," Mulford said. "That may have been a modest influence."

C. Fred Bergsten, who held the Mulford job in the Carter administration, recalled that when economic sanctions were imposed against Iran in 1979, there was great concern that there would be switching out of Arab dollar accounts, although little, in fact, took place.

"Tongue in cheek, you could say that freezing Libyan assets made the U.S. an 'unsafe' haven, because others may have sold dollars anticipating that Arab nations would do so," Bergsten said yesterday. Bergsten, now director of the Institute for International Economics, said that more basically, the dollar is still highly overvalued, and foreigners simply may have taken profits at a time of uncertainty.

He pointed out that logically, if the stock market slide on Wednesday had been precipitated, as reported, by fears that interest rates would firm up, the normal reaction in the foreign exchange markets would have been for the dollar to rise in value, instead of going down.

The basic impetus cited by most observers for the Wednesday slide in Wall Street stock market prices was a sizable improvement in December statistics for the economy. When this showed up in a larger employment total and a slight uenmployment reduction, market experts concluded they no longer could expect the Federal Reserve Board to trim interest rates.

In a telephone interview yesterday, Henry Kaufman of Salomon Brothers, who helped trigger Wednesday's decline in the Dow Jones index when he predicted interest rates wouldn't drop further, said:

"All I said was that the economy may be coming out of its lethargy. We may have a number of industrial production for December that could be up seven-tenths of a point. And personal income may be up almost a full point. On that basis, it's unlikely that the Fed would rush" to lower rates.

The bull market in Wall Street over the past few months for bonds as well as stocks has been fed by the expectation that the Fed would continue to help lower interest rates. Wall Street considers lower interest rates a bullish influence, because they tend to encourage expansion and increase profits.

In part, the general expectation of lower interest rates was traced to the successful passage of the Gramm-Rudman budget-reduction legislation; with a lower budget deficit, the Fed presumably would have room to allow interest rates to keep moving down.

But some influential voices in financial markets have begun to question that underlying assumption. For example, Ford administration economic adviser Alan Greenspan said yesterday that the financial world, once optimistic that the Gramm-Rudman Act would curb the budget deficit, now worries that it may be declared unconstitutional.