Senate Armed Services Committee Chairman Sam Nunn (D-Ga.), once an ally of the Reagan administration in its most critical military engagements on Capitol Hill, is emerging as a formidable administration adversary on a wide front including foreign policy and arms control as well as defense.

While mulling a race for the Democratic presidential nomination earlier this year, Nunn staked out a prominent role on arms control that put him in direct conflict with the White House and at the forefront of Democratic countermoves on arms policy.

Since eliminating himself as a 1988 presidential contender six weeks ago, Nunn has taken an even more assertive role in challenging President Reagan and his policies, lending his conservative credentials to Democratic efforts once spearheaded largely by more liberal colleagues.

Nunn, who gained influence from the skillful use of his expertise on military matters, has parted company with the administration before. And he remains more closely aligned with it on matters such as defense spending than most other Democratic lawmakers.

But his recent challenges, in areas ranging from the Persian Gulf to "Star Wars" defenses in space, present the administration with new foreign-policy problems just as it faces mounting threats to its major domestic goal for the year: Senate confirmation of Supreme Court nominee Judge Robert H. Bork.

Nunn's conflict with the administration came in the fiscal 1988 defense authorization bill that he steered to passage Friday after a bitter fight. By the end of the nearly five-month battle, which was interrupted by a summer-long GOP filibuster, Nunn had:

Led a successful effort to require congressional approval before Reagan can reinterpret the 1972 Antiballistic Missile (ABM) treaty to allow expanded testing and development of the Strategic Defense Initiative (SDI).

Thrown his weight behind numerical limits on strategic nuclear launchers that would force the United States back into compliance with the unratified SALT II treaty of 1979.

Pushed for enactment of modified war-powers restraints to require approval from Congress for long-term continuation of the U.S. tanker-escort operation in the Persian Gulf.

Used a variety of hardball tactics usually associated with more flamboyant lawmakers, including threatening to block consideration of Bork until the GOP stopped blocking the defense bill.

Moreover, the normally low-profile Georgian appeared to be enjoying the high-profile combat, saying his "blood was stirred" by GOP delaying tactics.

Acknowledging that he has become more combative, Nunn said he feels free of any "second-guessing" of his motives now that he is out of the 1988 race. He was also back on favorable turf where his reliance on facts rather than flair was a plus. Pressed unsuccessfully to provide some verbal flourishes for one of his victories, he smiled and said it was too much to ask of someone who had been described by a pundit as having the "charisma of a fish."

In forgoing the power that others are seeking in the presidency, at least for the time being, the 49-year-old Nunn seems content -- even eager -- to find it elsewhere. In his clashes with the administration on both arms control and the Persian Gulf conflict, he is moving to assert the powers of Congress, while expanding his own role in the exercise of those powers.

It is a test of wills that could dominate the waning days of the Reagan administration, influence next year's elections and affect the balance of power between Congress and the White House for future administrations. And, for Nunn, it could help determine whether he can overcome the apprehension among more liberal Democrats over his conservative record.

Two southern colleagues present different assessments of what Nunn is doing.

"He seems to be moving into a new role as a national Democrat, and it looks as though he's trying to shed some of the baggage that goes with coming from the South," said Sen. Thad Cochran (R-Miss.). "He may be getting in a position where he can be a legitimate presidential candidate sometime in the future."

It's not a question of Nunn moving to the left, said Sen. J. Bennett Johnston (D-La.), who credits administration policies and tactics for pulling Democrats together. "I look at my own record and it begins to look pretty liberal by Louisiana standards," said Johnston, who has gone further than Nunn in advocating SDI cuts.

Nunn also blames the administration for any parting of ways.

"They {administration officials} are taking the position they can do anything they want without Congress. That's intolerable to me," he said in an interview.

"I don't go looking for fights, but when someone forces me into a fight, I don't walk away . . . . They're very cooperative so long as you agree with them. Their idea of bipartisanship is doing it all their way," he said.

As recently as last year, the administration could rely on Nunn to back the Senate Republican leadership in opposing arms-control assaults from the more liberal, activist House.

But this year it lost its first line of defense when Democrats took over the Senate, and it now finds Nunn in agreement with the House on two of its major arms proposals: the SALT II weapons limits and SDI testing constraints.

Some Republicans have expressed concern that Nunn could use approval of the SALT II limits as bargaining leverage to force administration acceptance of the ABM-SDI provisions, which have been his main concern.

While Nunn's role on arms control has drawn praise from more liberal Democrats, he faces what could be an even more critical test for acceptance by the mainstream of the party: his vote on the Bork nomination.

Even as many of his southern Democratic colleagues were announcing their opposition to the conservative jurist, Nunn was brushing aside questions about his position, saying he had enough to worry about in the defense bill and had not made up his mind about Bork.

But, if Bork has a chance of confirmation, it could well turn on just a few people from both parties, putting enormous pressure on Nunn to stray even further from his conservative past if he still harbors national ambitions.

"It will say more than anything else whether he really has presidential ambitions," said one of Nunn's colleagues, asking not to be quoted by name. "No Democrat can vote for Bork and expect to be considered for national office," he added.

Thomas E. Mann, former executive director of the American Political Science Association and now with the Brookings Institution, put it another way: "I really believe that if he opposes Bork, those core Democratic groups will forgive him" for previous votes against voting rights and other liberal legislation, said Mann. Nunn has passed muster on foreign policy but still has problems on domestic issues, he added.

"This is the vote of an era. If he goes with them on this, he's put himself on the playing field for national office." Mann concluded.