President Reagan asked House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) yesterday for permission to address the House before its vote this week on an aid package for the Nicaraguan rebels, but O'Neill refused, saying such a speech would be "unprecedented."

The president's gambit and O'Neill's rebuff were part of the final jockeying before Wednesday's scheduled vote on modified versions of Reagan's request for $100 million in aid to the rebels, a vote that both sides predict will be extremely close. Presidential spokesman Larry Speakes said Reagan was "deeply disappointed" that O'Neill refused his request.

O'Neill said he made a counteroffer that the president speak before a joint session of Congress, but White House Chief of Staff Donald T. Regan, speaking on the president's behalf, turned it down. O'Neill said "the only justification" possible for the "unorthodox procedure" of Reagan coming before the House alone would be if Reagan would take questions and answers.

Speakes said the idea of a speech to a joint session of Congress was rejected because the Senate is occupied with the tax-overhaul legislation and the House is "who we want to talk to."

Although the proposal for a Reagan appearance before the House was first suggested to O'Neill yesterday, Speakes said it had been discussed within the White House for several days, and Reagan had been working on a near-final draft of his planned speech. Officials said Reagan would submit the speech to Congress Tuesday.

Speakes said Regan telephoned O'Neill with the request about 3 p.m. yesterday. The president did not make a personal appeal to the speaker. Speakes said the White House "never anticipated" O'Neill might reject the request, and a nationally televised address, which Reagan has attempted before on the eve of crucial votes, was not under consideration.

Both White House officials and Democratic strategists on Capitol Hill said the Reagan maneuver and O'Neill's response were unlikely to affect the outcome of Wednesday's vote. Stephen K. Patterson, press secretary to Rep. Dave McCurdy (D-Okla.), who heads a key group of moderate Democrats considered swing votes on the issue, said, "This issue has been debated so thoroughly over the last 20 months it's hard to believe there will be any new information. We have a package ready to go that we think will have a majority. We don't anticipate a sudden turnabout because of this event today."

White House officials taunted O'Neill, passing to reporters an excerpt from Article Three of the Constitution that says the president may "on extraordinary occasions, convene both Houses, or either of them." Reagan had no plans to invoke this power, however, they said. Republican National Chairman Frank J. Fahrenkopf Jr. characterized O'Neill's rebuff as "naked political arrogance."

In a written statement, O'Neill said his offer for Reagan to speak to a joint session remains open. "On Wednesday, the House votes on contra aid for the third time this year," he said. "If the House passes contra aid in any form, the Senate will have to act on the matter. Since future congressional action must occur in both houses, I believe that the proper forum for an address is the traditional joint session."

O'Neill recalled that President Nixon had appeared before both Houses sequentially on the eve of Vietnam war protests to thank members for supporting his conduct of the war. "Having the president appear before only one House to lobby for a legislative proposal would be unprecedented," he said.

The House Rules Committee is to meet today to adopt ground rules for the debate. The McCurdy plan, which the administration strongly opposes, would provide $30 million in nonmilitary aid to the contras immediately and make the $70 million in military aid the administration is seeking subject to a second congressional vote to be held on or after Oct. 1.

House Republicans are expected to counter this with an amendment to be offered on the floor Wednesday that would provide about $12 million in nonlethal "humanitarian" aid immediately and about $28 million in military assistance on Sept. 1. The GOP plan, which calls for additional installments of $20 million on Oct. 15 and $40 million on Feb. 15, would not be subject to a second congressional vote. Mandating a second vote of approval for the aid to continue has become a key issue.

On March 20, the House voted 222 to 210 to reject Reagan's original request for $70 million in military aid and $30 million in "humanitarian" assistance for the contras.