The Reagan administration, bowing in advance to Senate conservatives, said yesterday that it will accept a limit on the jurisdiction of the International Court of Justice to gain their support for ratification of a long-stalled treaty against genocide.

In testimony before the Senate Foreign Relations Committee, Elliott Abrams, assistant secretary of state for human rights and humanitarian affairs, said the administration has decided that a limitation is the "wiser course" and is the "only way" to gain Senate backing for a treaty that remains unratified after 37 years of debate and approval by 96 other nations.

Last September, President Reagan unexpectedly said that he would "vigorously support" ratification and that he intended to use it "in our efforts to expand human freedom and fight human-rights abuse around the world."

The concession on the treaty's Article IX, apparently arranged beforehand with Committee Chairman Richard G. Lugar (R-Ind.), immediately sparked sharp criticism from other senators, notably Christopher J. Dodd (D-Conn.).

Dodd said that it would gut the U.S. commitment to the treaty of any meaning and undermine U.S. efforts to seek prosecution of "the Idi Amins of the world," a reference to the former Ugandan dictator.

"It's the Nicaragua case. It's the Nicaragua case," Dodd shouted repeatedly at Abrams, State Department legal adviser Davis Robinson and acting assistant attorney general Ralph Tarr, who argued the administration's case for approval.

Dodd was referring to the administration's refusal last spring to accept the court's jurisdiction when Nicaragua brought a case against the United States for mining Nicaraguan harbors.

Abrams and his colleagues conceded that Nicaragua's lawsuit was one rationale, but indicated that the conservatives' longstanding treaty opposition, led by Sen. Jesse Helms (R-N.C.), was just as important in administration calculations.

After the Foreign Relations Committee voted 17 to 0 in favor of the treaty last September, Helms singlehandedly stalled a full Senate vote before the 98th Congress adjourned. In committee, he had voted "present."

Article IX states that disputes will be submitted to the court, also known as the World Court, "at the request of any of the parties." It was not immediately clear whether the administration's agreement to accept a "reservation" on the article would be sufficient to change Helms' position.

Helms appeared moved, however, by a highly emotional plea for ratification from Elie Wiesel, head of the Holocaust Memorial Council and a survivor of the Auschwitz concentration camp.

Congressional sources and State Department officials said that the wording of the "reservation" must be worked out. An aide to Lugar said it probably would be modeled on one adopted by India, which stated that both parties must agree in advance before submitting disputes to the court.

A State Department source said that the administration has not endorsed "any specific language" and is willing to work "toward a consensus" with the Senate committee.

Of the 96 nations that have approved the treaty, 20 have entered reservations, including 11 communist countries. Britain and Australia have questioned the validity of those countries' formal ratification and acceptance of reciprocity with them.