The Supreme Court of Warren E. Burger, 12 years in search of a mission, may have found one in the term that ended yesterday.

Its major rulings show a firm commitment to a drastically curtailed role for the federal judiciary as a check on the rest of government. If Congress wants to draft only men, the court said this term, the court must defer to Congress. If the executive wants to deny freedom to travel abroad, the court must defer to the executive.

If the states have overcrowded prisons, or terrible institutions for the mentally retarded, the court must defer to the states.

The theme has been developing ever since the backlash to the court's 1973 legalization of abortion. It reached its hight this year, particularly when confronted with foreign policy, national security or military questions. And it will undoubtedly flourish as the Esienhower, Kennedy, Nixon and Ford court becomes, as well, a Reagan court.

It is a humble court, finally a Burger court, not an Earl Warren court. It is a deferential court that "knows its place" in the scheme of government, not a court that tries to carve a place.

It means that elections are more important than ever. When it comes to questions of social change, the message increasingly is: don't bother to file a suit. Vote, lobby or make a campaign contribution. The justices of the Supreme Court are increasingly saying, "Who are we to question?"

This is part of what conservatives and President Reagan have been wanting from the federal judiciary for years. But it cuts many ways.

The same "deference" to Congress that upheld the all-male draft was used 12 months ago to uphold affirmative action in the award of government contracts, and a few weeks ago to uphold tough federal restrictions on the strip-mining industry.

There are, of course, going to be exceptions. Demonstrating that it still knows how to hold something unconstitutional, the court this year struck down a zoning ordinance used to ban nude dancing because the ordinance was too broadly restrictive of free expression. But in borough of Mount Ephraim, N.J., not the Congress of the United States, was the victem of that ruling.

The court did take on Congress on one issue: judicial salaries. The justices ruled this year that Congress acted unconstitutionally on two occasions when it denied pay raises to federal judges.

Six months after ruling that poor women had no constitutional right to money for abortions, the court said justices had a constitutional right to their pay raises.

There are lots of contradictions. The case of former CIA agent Phillip Agee is an example. The court ruled that the executive branch could take away his passport, even though Congress had said nothing about the issue.

This, however, was enough for the court.In the field of institutions for the handicapped, however, Congress has enunciated a relatively clear prescription for protecting patients from mistreatment and isolation in institutions.

But this term, the court said in a case involving Pennsylvania's Pennhurst home for the retarded, what Congress said wasn't enough to require states to remedy the situation.

The justices had a variety of explanations for their actions this year, and many of them sounded like Reagan campaign speeches.

Here is Justice Lewis F. Powell Jr. ruling that double-celling of state prisoners is constitutionally acceptable in Rhodes v. Chapman: "courts certainly have a responsibility to scrutinize claims of cruel and unusual confinement. . . . However, courts cannot assume that state legislatures and prison officials are insenitive to the requirements of the Constitution. . . ."

Here is Justice William H. Rehnquist ruling in Rostker v. Goldberg that Congress may exclude women from the draft: "The Congress is a co-equal branch of government whose members take the same oath we do to uphold the Constitution of the United States."

And here is Burger, in the Agee case: "Matters intimately related to foreign policy and national security are rarely proper subjects for judicial intervention. . . . Matters relating to the conduct of foreign relations are so exclusively entrusted to the political branches of government as to be largely immune from judicial inquiry or interference.'"

The court did not need the election to come up with this language, however. The final words in the passage from the Agee case come from a 1952 opinion authorizing deporation of resident aliens who were once members of the Communist Party of the United States.

Justices explain their opinions. But they rarely explain publicly what might be going through their minds. When Potter Stewart announced his retirement, he offered a rare glimpse of what was on his mind at the mement.

Congressional proposals to strip the court of jurisdiction over controversial issues, such as abortion, school prayer and busing, "concern me," Stewart said. "There have been such bills in Congress ever since I've been here . . . but there seems to be considerably more of a possibliity that one or more of such bills might be enacted."

The justices are not deaf to the outcry from large segments of the public over decisions in the past. The court's rulings this year on the draft, upholding the constitutionality of parental notification of abortions, last year's approval of the Hyde Amendment, which prohibited the use of federal funds for abortions in virtually all circumstances, and its general posture may help defuse that outcry.

The sex discrimination rulings -- particularly in the draft case and in a decision upholding laws that make statutory rape a crime for men but not women -- illustrate the context.

A decade ago, the Supreme Court began making changes in the sex discrimination laws that were barely noticed by the world. A state couldn't let younmg women buy strong beer but deny it to young men, the court said. Women could not be prohibited from administering wills, it ruled.

The changes were subtle. The gender distinctions were relatively innocuous. But it was clear that they wouldn't stay that way, for the inevitable result of the court's reasoning would be confrontations with sensitive and basic views held by many about the role of women in society.

The early cases came before the abortion uproar. In the case of whether women must be included in the draft, another one loomed.

In that case and the statutory rape case, Michael M. v. Superior Court of Sonoma County, the court backed away from its course in sex discrimination laws, and women's rights lawyers are now speculating that it may be a permanent backing away.

The court essentially ignored the legal principle it had established allowing prior gender distinctions to fall: the principle that any distinction required thorough justification based on facts, not sterotypes.

In the early case, Craig v. Boren, for example, the court said it was not enough for the state of Oklahoma to justify selling strong beer to women and light beer to men by saying that men have a higher incidence of drunken driving. The court examined that claim and its validity, and found it wanting.

In the draft case, by contrast, the court said women could be excluded because they were ineligible for combat. But, it did not examine the validity of the combat exclusion.

Instead, the court seems to be saying that different treatment of men and women can be justified as long as Congress enacts it into law.