NEW YORK, DEC. 31 -- The stock market closed out 1987 on a cautious note in the face of renewed weakness in the dollar.

The Dow Jones average of 30 industrials slipped 11.27 to 1938.83, finishing the year with a net gain of 42.88 points, or 2.26 percent.

Some broader market measures came in below their levels at the end of 1986.

So a wild year that produced 55 record highs in the Dow and then the steepest drop in modern times wound up as "mixed."

The stock markets in Tokyo and many European cities were closed for the year-end holiday.

But in London, the benchmark Financial Times-Stock Exchange index fell 47.1 points to close at 1,712.7. That index is down nearly 30 percent from its all-time peak in mid-July.

In Paris, share prices retreated by more than 1.5 percent on the Bourse, giving them a 28.1 percent average decline for the year.

Advancing issues slightly outnumbered declines in the daily tally on the New York Stock Exchange.

Big Board volume totaled 170.14 million shares, up from 149.28 million in the previous session.

As the business day began in New York, traders were confronted with the news of a drop in the dollar to new lows in currency markets around the world.

Analysts said there also seemed to be a last-minute round of stock selling by individuals for tax purposes, and by money managers readying their portfolios for year-end reports to clients.

Such selling was offset earlier in the month by bargain-hunting traders hoping to profit from a January market rally.

Lately, however, many analysts have questioned the chances for a substantial rebound in the next few weeks.

After the early selling, prices staged a weak recovery, with many smaller issues attracting better support than the blue chips in the Dow.

Much of the day's activity was concentrated in high-yielding utility issues with quarterly dividend dates near.

Public Service of Colorado, down 1/8 at 20 3/8; Southwestern Bell, up 5/8 at 34 3/8; Central Maine Power, down 1/4 at 15 1/2; Oklahoma Gas & Electric, unchanged at 29 1/4, Delmarva Power & Light, up 3/8 at 18; and Boston Edison, declined 1/4 at 18 3/4, ranked as the six most active NYSE issues.

Losers among the blue chips included Du Pont, down 1 at 87 3/8; Eastman Kodak, down 1 at 49; Ford Motor, down 1 1/8 at 75 3/8; International Business Machines, down 1 1/2 at 115 1/2, and International Paper, down 3/8 at 42 1/4.

In the technology sector, which was strong on Wednesday, Digital Equipment dropped back 2 3/8 to 135; Hewlett-Packard 1/2 to 58 1/4; Honeywell 1 3/4 to 53 7/8, and National Semiconductor 1/4 to 12.

As measured by Wilshire Associates' index of more than 5,000 actively traded stocks, the market lost $3.05 billion, or 0.13 percent, in value.

The NYSE's composite index of all its listed common stocks dipped .30 to 138.22.

Nationwide turnover in NYSE-listed issues, including trades in those stocks on regional exchanges and in the over-the-counter market, came to a total of 206.98 million shares.

Standard & Poor's index of 400 industrials dropped 1.20 to 285.86, and S&P's 500-stock composite index was down .78 at 247.08.

The Nasdaq composite index for the over-the-counter market rose 0.77 to 330.47.

At the American Stock Exchange, the market value index closed at 260.35, up 3.00.