The stock market registered its sourth consecutive decline yesterday as volume slowed to its lowest level in nearly two months.

Analysts said interest rate worries, including a rapidly spreading prime rate hike, concern about inflation and a larger than expected April trade deficit contributed to selling by investors.

The Dow Jones 30-stock industrial average dropped 3.72 points to 831.69. After an 8.57 gain on Monday, the market dropped 23.73 points over the next four days for a decline for the week of 15.16 points. This was the largest one-week retreat for the market since mid-February when the Dow fell 23.30 points.

Volume on the New York Stock Exchange was a comparatively light 21.4 million shares yesterday, the smallest daily turnover since April 4 when 20.1 million shares changed hands. On Thursday 28.4 million shares traded.

The light level of trading was attributed to an early start to the three-day Memorial Day holiday weekend.

'After five weeks of very heavy volume, some catching of breath is to be expected and is probably necessary," commented Eric Miller, head of the investment policy committee at Op-penheimer & Co.

A factor in the market's drop was the action by many large commercial banks in raising the prime interest rate to 8 1/2 percent, following the lead of Continental Illinois National Bank of Chicago which initiated the quarter percent increase late Thursday.

Citibank was the only one of the country's 10 largest banks which as of yesterday had failed to raise the prime rate.

There also was concern that food prices for 1978 could rise substantially more than the 6 to 8 percent predicted by the administration, adding to fears of accelerating inflation.

And the April trade deficit of $2.86 billion was the fourth largest on record, apparently larger than investors had anticipated. This is turn caused the dollar to weaken somewhat in foreign exchange trading.

Some analysts said the market continued to show fundamental strength, despite the four-day setback. And they predicted that stocks could resume their upward momentum by next week.

"We feel that the market has been going through a digestive phase here," said Hildegard Zagorski, a market analyst with Bache Halsey Stuart Shields.

"It has moved sharply higher since the beginning of April and was in need of some rest and recuperation.

The digestive phase is probably over and sometime next week the market should resume its rally."

Zagorski said the market's next resistance level is 860 on the Dow, and if it breaks through that barrier "the next target is 900."

"The market had a very big obstacle in the prime rate increase, and, considering that, it's doing pretty well,? commented Bud Simons, vice president with Cowen & Co.

"Tuesday morning investors will get the consumer price index for April which, evidently, is going to be a terrible one," added Simons, but he predicted that the market would probably move upward once the April inflation numbers are out of the way.

The NYSE composite index of its total list of common stocks declined 0.10 to 54.14. And the Standard & Poor's 500-stock composite index was down 0.22 to 96.58.

The most active stock on the NYSE was Arlen Realty & Development which fell 1 5/8 to 3 3/8 after the company estimated a $100 million lost for its fiscal year, due to writedowns of its portfolio.

Union Oil of California also was active as a 219,000 share block traded at 50. For the day it was unchanged at 49 3/4.

On the American Stock Exchange, the market value index rose 0.33 to 144.30.

The NASDAQ composite index for the over-the-counter market gained 0.04 to 119.86.