The Reagan administration told Congress yesterday it is considering withdrawing from the International Atomic Energy Agency as a result of the rejection last Friday of the credentials of the Israeli delegation to the agency's annual meeting.

"We will study the entire range of options, not excluding withdrawal," Deputy Assistant Secretary of State James B. Devine told the Senate Foreign Relations Committee. "We will also look at reducing our funding and other support for agency activities."

While administration sources suggested privately that a permament U.S. withdrawal from the atomic energy agency, the only world body that attempts to deter the misuse of nuclear materials, is unlikely, they suggested that a punitive cut in American backing of the agency was a real possibility.

Beyond the fact that the United States currently contributes about $33 million to the agency, or about 27 percent of its annual budget, U.S. technical experts play a major role in assisting the organization, particularly in improving its instruments and procedures for safeguarding nuclear facilities.

The Reagan administration, in fact, has made improving the agency's ability to detect any diversion of material from civilian atomic facilities for possible use in nuclear weapons the cornerstone of its non-proliferation policy.

As a result, Devine told worried senators yesterday that while the United States will reduce its participation in agency activities until the reassessment is completed, "we will attempt to ensure that its safeguards-related functions do not suffer, given the importance safeguards have to the United States."

But he said the United States, which walked out of the agency meeting following the vote to bar Israel for "genocide perpetrated against the Palestinian people," was determined to prevent further politicization of the U.N. body.

Since Friday's walkout, the United States boycotted a meeting Saturday of the agency board of governors, and does not intend to send representatives to committee meetings that will be held in the next two weeks.

The U.S. Mission in Vienna also has been instructed to keep contacts with the agency secretariat to a minimum, a State Department source said. However, he added, the United States will send observers who will take notes but not participate in any sessions dealing with safeguards issues.