NEW YORK, DEC. 30 -- A steadier dollar helped the stock market stage a mild rally today in an atmosphere of continuing caution as the end of 1987 approached.

The Dow Jones average of 30 industrials, down a total of 72.78 points on Monday and Tuesday, advanced 23.21 today to close at 1950.10.

With one session remaining in a year of wild swings, the Dow shows a net gain of 54.55 points, or 2.9 percent.

Advancing issues outnumbered declines by more than 2 to 1 in today's activity on the New York Stock Exchange. Big Board volume totaled 149.28 million shares, up from 111.58 million in the previous session.

Stocks seemed to draw some strength from a steady showing by the dollar in foreign exchange.

Otherwise, analysts said it was difficult to read much of significance into the market's recent fluctuations, given strong seasonal influences on trading.

In the past couple of days, they said, it was apparent that some investors were still selling stocks for tax purposes or to ready their portfolios for year-end reports to clients.

On the other side of the coin, some traders have been buying lately in hopes that depressed issues might rally after those year-end pressures are lifted.

In the economic news, the Commerce Department reported today that the index of leading indicators fell 1.7 percent in November. A substantial drop had been widely expected.

Gainers among the blue chips included International Business Machines, up 1 at 117; American Telephone & Telegraph, up 3/4 at 27 1/2; General Electric, up 7/8 at 45 1/8; Merck, up 3 at 158 7/8, and Coca-Cola, up 1 1/8 at 38 5/8.

J.P. Morgan, which was involved in developing a bond-swap plan to deal with Mexico's debt problems, rose 1 1/2 to 36 5/8, and some other money-center bank issues also advanced.

But Manufacturers Hanover fell 1 3/8 to 22 1/8. The company said that it would report a $23.5 million aftertax gain in the fourth quarter from the sale of stock in its Portuguese subsidiary.

Mexico Fund, a closed-end investment company concentrating on the Mexican stock market, climbed 5/8 to 4 3/8 to register one of the day's best percentage gains.

Several leading technology stocks also gained ground. Digital Equipment added 3 1/8 to 137 3/8; Hewlett-Packard 1 5/8 to 58 3/4, and Texas Instruments 1 3/8 to 56 1/2.

As measured by Wilshire Associates' index of more than 5,000 actively traded stocks, the market rose $30.57 billion, or 1.28 percent, in value.

The NYSE's composite index of all its listed common stocks picked up 1.68 to 138.52.

Nationwide turnover in NYSE-listed issues, including trades in those stocks on regional exchanges and in the over-the-counter market, totaled 181.42 million shares.

Standard & Poor's index of 400 industrials rose 3.91 to 287.06, and S&P's 500-stock composite index was up 3.27 at 247.86.

The Nasdaq composite index for the over-the-counter market gained 4.17 to 329.70.

At the American Stock Exchange, the market value index closed at 257.35, up 2.07.