Secretary of State Cyrus R. Vance has resigned his post after failing to persuade President Carter not to use military force in the hostage crisis with Iran, informed sources said last night.

Vance, considered the most loyal as well as the most prestigious member of the Carter Cabinet, consistently opposed the U.S. military operation that tried and failed to rescue the American hostages in Tehran. He also opposed the wider application of U.S. military force in the region, the sources said.

Vance actually presented Carter with his decision to resign last Monday, while final preparations were being made for the rescue mission he opposed. Carter asked him to reconsider, but Vance refused.

The resignation was withheld until after the mission had been completed because Vance felt it would be "unseemly" to let it be known earlier, according to an informed source. The resignation furor might also have revealed the secret military plan. An official emphasized that Vance would have resigned even if the mission had been a success.

Lights were on late at the Vance residence in Northwest Washington last night, but a State Department security aide said the secretary would have no immediate statement.

Vance is the first secretary of state to resign as a matter of principled protest in 65 years -- since William Jennings Bryan resigned from Woodrow Wilson's Cabinet in 1915, protesting Wilson's policies toward Germany and the war in Europe.

Deputy Secretary of State Warren Christopher, an old associate of Vance's who served as his confidant as well as deputy, will serve as acting secretary of state for the time being, according to official sources.

Carter is reported determined to "move quickly" to name a successor. Christopher and White House Counsel Lloyd Cutler are among those considered most likely to be selected. Presidential assistant Zbigniew Brzezinski has also been mentioned as a candidate for the post, but is unlikely to be named under present circumstances. Brazezinski was Vance's opponent in many policy disputes, including the one which triggered the surprise resignation.

Vance's departure is likely to prompt a new crisis of confidence in Carter's leadership abroad as well as at home. Vance was considered by foreign leaders in western nations as well as the Soviet Union to be the exponent of stability and prudence in a government of baffling reverses. His departure at this moment may complicate the administration's efforts to line up allied support for policies of pressure on Iran and on the Soviet Union.

At home, Brzezinski was often considered more influential -- as in a recent U.S. News poll -- but Vance was vastly more popular, winning votes of confidence from Congress and elements of the public worried by Brzezinski's policies.

The knowledge of Vance's attitude on the rescue mission, which spread through Washington political circles over the weekend, revealed a major breach in the administration ranks on the risky and ill-fated operation.

In recent days, administration officals took pains to conceal the policy dispute, and Vance, ever the team player, remained out of public view in order to avoid revealing his disagreement.

Whenever possible, officials evaded the question. For example, Brzezinski was asked on "Issues and ANSWERS" (ABC, WJLA) to amplify reports that Vance had "very grave doubts" about the rescue operation.

Brzezinski avoided commenting on Vance's attitude, but said, "Everyone recognized that the operation was risky." Brezezinski went on to say that President Carter took "the right decision and the courageous decision" after extensive discussions in the National Security Councel "in which all of the president's advisers took part."

Another senior White House official, asked if the decision to proceed with the mission had been unanimous among the president's advisers, replied that "the National Security Council is not a democratic body." When pressed, he would say only that the mission was "constantly discussed" among high officials.

White House press secretary Jody Powell, confronted with similar questions Saturday, replied that "I have not indicated that everyone went along with it." Powell told reporters "there were serious questions all along . . . even after the mission was initiated there were serious questions raised about this and that." He did not elaborate.

Vance did not articipate in the briefings for foreign ambassadors or reporters Friday in the wake of the announcement of the rescue mission and its disastrous conclusion. He left his office earlier than usual Friday. aAdies said he was suffering from an attack of gout, which kept him home over the weekend.

According to an official chronology, the basic decision by Carter to proceed toward a military effort to free the hostages was taken at a National Security Council meeting April 11. Vance was out of town at the time on a long-planned vacation weekend in Florida.

Vance reportedly was rankled by the taking of such a momentous decision in his absence as well as by the substance of the decision. This is not the first time that big decisions in the foreign policy field were made while he was away from Washington.

The final agreement on establishment of U.S. diplomatic relations with the People's Republic of China came while Vance was in the Middle East in December 1978 -- and probably doomed the expected completion later than month of the strategic arms limitation treaty he was negotiating with the Soviet Union.

Vance's influence within the administration waned noticeably after the controversial incident that saw the United States on March 1 vote for a United Nations Security Council resolution criticizing Israel only to have Carter publicly disavow the vote two days later.

The administration subsequently explained there had been a misunderstanding between Carter and Vance that caused the president to be unaware that the resolution contained references to the status of Jerusalem that he considered unacceptable. Vance publicly accepted responsibility for what was termed a "failure in communications."

Later in March, Vance, testifying before Congress about the incident, said the administration still stood behind that part of the resolution that did not mention Jerusalem. While that was an accurate statement of administration policy, it was made on the eve of the New York Democratic primary, and Carter's campaign strategists subsequently blamed the president's defeat there by Sen. Edward M. Kennedy (D-Mass.) partly on the anger that Vance's testimony caused among New York Jewish voters.

In the aftermath of the New York vote, White House officials became increasingly more open in hinting to reporters that Vance might be turning into a liability for Carter's reelection hopes. About the same time, a parrallel lack of influence by the State Department, particularly in regard to Middle East affairs, started to become evident.

That was revealed most glaringly during the mid-April visit here of Egyptian President Anwar Sadat and Israeli Prime Minister Menachem Begin for talks with Carter about salvaging the stalled negotiations over Palestinian autonomy.

Although Vance took part, he and the State Department, which had been branded as a hotbed of "Arabists" by Israel's supporters, clearly were out on the fringes of these talks.

Similarly, it was about the same time that the administration began to shift perceptibly away from the moderate, step-by-step approach toward Iran -- an approach of which Vance had been the leading advocate -- toward more open bellicosity that included increasingly broad hints of possible military action, which preceded last week's abortive rescue attempt.

Vance's dissent from the rescue mission became known as the Carter administration edged close to a public commitment to consult U.S. allies before undertaking broader military action against Iran, such as blockading or mining in the Persian Gulf.

Brzezinski and Secretary of Defense Harold Brown, in separate television interviews yesterday, stopped just short of a full-scale pledge of consultation with the allies, which are reported to be strongly opposed to such U.S. operations.

Brzezinski drew a distinction between a "rescue mission" and a military action of a more sustained nature. A sustained action, he said, "would lend itself to more sustained consultation" with allies.

Brown responding to questions on "Face the Nation" (CBS, WDVM), said U.S. allies expected to be consulted or informed in advance of such actions as blockading or mining Iran's harbors. He maintained that "all of the allies" understood why consultation was impossible before the surprise rescue mission.

State Department sources said ambassadors of allied countries were given indications in meetings with Christopher on Friday that their governments would be consulted before any blockade-mining actions. This was described as the "consensus" within the executive branch rather than an explicit administration decision.

Remarks of high administration officials in recent days have also strongly suggested that Congress will be fully consulted before any broadscale military action.

In a related development, the White House made public a presidential report to Congress on last week's unsuccessful rescue mission consistent with the reporting provisions" of the War Powers Act. The document provided little detail beyond that which already had been made public. President Carter did say that "no United States armed forces remain in Iran."

Both Brown and Brzezinski pointedly declined to rule out blockading or mining -- or any other military action -- as a possibility in the hostage crisis. Brzezinski went out of his way to stress that Iran should know that the United States will "do what is neccessary" to obtain release of the hostages.

Speaking in sharp tones, the national security affairs adviser announced what he called "a very important message" to Tehran: "Do not scoff at American power. Do not scoff at American reach. It is in Iran's interest to resolve this matter peacefully."

Adked if the United States would "go to war with Iran" over the hostage issue, Brzezinski replied: We will take actions which are appropriate. We prefer to solve this problem peacefully through collective pressure. We hope -- as I have said many times in this program -- the Iranians will reach the right conclusion."