The Reagan administration disagreed yesterday with Israel's claim that Iraq had been on the verge of building nuclear weapons, but it added that the United States had been concerned about that posibility before Israel bombed the Iraqi nuclear reactor last week.

Testifying before a House Foreign Affairs subcommittee, Undersecretary of State Walter J. Stoessel Lr. said: "We don't agree with that position of Israel . . . . We have not made any definite conclusions" that the Iraqis "were working toward" a nuclear weapons capability."

His remards came as both houses of Congress began inquiries into whether Israel violated agreements with this country by using American-supplied weapons for offensive purposes. Stoessel, William Casey, director of the Central Intelligence Agency, and other administration officials are scheduled to testify before the Senate Foreign Relations Committee today.

While Stoessel diverged yesterday from the Israeli position on Iraq's ability to produce an atomic weapon, he stressed that the United States had been worried about Iraq's nuclear program and the tensions it had introduced into the Middle East situation.

In that resepct, his remarks appeared to follow the line laid down by President Reagan at his Tuesday press conference. The thrust of Reagan's remarks was that, while he regretted Israel's failure to seek a diplomatic solution, he felt that the Jewish state was justifiably concerned about its security in the face of threats from its Arab neighbors.

Although a finding by Reagan or Congress that Israel used its U.S.-supplied jets in violation of U.S. law could lead to a cutoff of aid, some key members of Congress made clear yesterday that such action seems unlikely.

Rep. Lee Hamilton (D-Ind.), chairman of the House Mideast subcommittee, said: "I do not expect Congress to take action against Israel." Similarly, Chairman Charles H. Percy (R-Ill.) of the Senate Foreign Relations Committee said on Nbe television's "Today" show: "I don't think Congress would ever cut Israel off and leave it to the tender mercies of its adversaries in the Middle East."

Percy added, though, that he thinks Israel will have to make some concessions to win release of the for F16 jet fighters whose delivery was held up by Reagan following the raid. The senator did not specify what these concessions might be, but he predicted that Congress will "send a message to Israel that taking precipitous action of this kind, which vitally affects American policy, cannot be unilaterally undone."

Further questioning of the Israeli claim that Iraq was close to producing a nuclear weapon came yesterday in a report prepared by the Congressional Research Service of the Library of Congress. It said that for Iraq to "misuse" its reactor woudl have required a series of "telltale" technical and physical changes that would have been detected by international inspectors and French technicians helping to construct the reactor.