The unexpectedly large increase in oil prices by petroleum-exporting countries sent stock prices tumbling in heavy trading yesterday. The Dow Jones average of industrial stocks fell 17.84 points.

The bigger-than-expected oil price rise, which Federal Reserve Board Chairman G. William Miller conceded will make inflation harder to fight at home, also hit the nation's bond markets and depressed the dollar on foreign currency markets.

Most private and government analysts had expected the Organization of Petroleum Exporting Countries to announce a price increase in the range of 5 to 10 percent, effective January 1.

Instead, the OPEC countries, announced a 5 percent rise on crude oil prices Jan. 1, with successively smaller increases on April 1, July 1 and Oct. 1.

The drop in the Dow Jones average yesterday was the biggest one-day decline since Oct. 31, the day before President Carter announced a series of actions to prop up the value of the dollar.

Both the dollar and domestic stock prices surged after Carter actions.

Yesterday's steep decline in stock prices-the New York Stock Exchange Index fell 1.11 to 52.20 and the price of an average stock declined 62 cents-mainly reflected invester concern that the OPEC oil price increase will hurt the nation's anti-inflation fight, worsen the trade balance and keep the dollar from recovering.

Each percentage point rise in oil prices adds $1 billion to the nation's oil import bill and tosses more unwanted dollars into the hands of foreigners.

Newton Zinder, an analyst for E. F. Hutton, said few other factors than the OPEC price hike affected yesterday's heavy selling. But Zinder said that mose of the impact was probably felt yesterday.

Whenever the market faces a "negation surprise," Zinder said, "we usually find most of the reaction occurring on the first day." But Zinder said it would not be fair to conclude that the market will rebound in the next few days.

Federal Reserve Board chairman G. William Miller said he was not surprised by the reaction of the foreign exchange and stock markets, but thought that the markets were reacting emotionally and not looking at the fundamentals.

Of the issues traded on the New York Stock Exchange yesterday, only 127 gained ground, while 1,576 declined in value.

The stocks that would be most hurt by a rise in oil prices, such as airlines and companies whose profits are geared to automobile travel, were hit hard.

Delta fell 2 1/2 to 40 1/4. Holiday Inns fell 1 3/4 to 17, UAL declined 1 1/2 to 29 1/8 while Ramada Inns lost a half point, closing at 8 1/4.

All told, there were 35,382,328 shares of NYSE-listed stocks traded on all U.S. securities exchanges, compared with 25,041,428 last Friday.

The Standard & Poor's index of 400 industrial stocks fell 2.10, closing at 103.83. The american Stock Exchange index fell 3.62 to 145.68 while the NASDAQ index of over-the-counter stocks fell 3.08 to 114.33.