The Senate yesterday easily agreed that Congress should have some control over the duration of the U.S. military occupation of Grenada, then exploded into impassioned debate on the wisdom and legality of President Reagan's military policies around the world.

The four-hour outburst on foreign policy, which came up suddenly when the Senate was supposed to be working on a debt bill, featured a blistering attack on Reagan from maverick Republican Lowell P. Weicker Jr. of Connecticut.

As Majority Leader Howard H. Baker Jr. (R-Tenn.) proposed an amendment commending Reagan for the Grenada invasion, Weicker leaped up to roar: "It might be one thing to turn your head when the president of the United States violates the law, but I'm not going to commend him for it!"

That ignited Baker, normally one of the Senate's coolest customers, into an equally heated reply. In the midst of his fervent defense of the U.S. move into Grenada, however, Baker also revealed that he wrote Reagan a month ago to express "grave reservations" about the deployment of U.S. Marines in Lebanon.

In the House, meanwhile, Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.), who has supported Reagan on Lebanon, blasted the Grenada invasion as a "frightening" adventure in "gunboat diplomacy."

O'Neill had called on Democrats not to attack the president on Grenada as long as American soldiers might be under fire. But yesterday he effectively declared open season on the issue. "The policy is wrong," he declared. "Mr. President, please don't continue your policy."

The Senate debate on foreign policy began with a relatively mild discussion of the applicability of the War Powers Resolution of 1973 to the Grenada invasion. With most Republicans going along, the Senate agreed on a 64-to-20 vote that the law, which requires congressional approval if troops are left in hostile situations overseas more than 60 days--Dec. 24 in the case of Grenada--applies to the Caribbean operation.

But that vote prompted Baker to introduce his offsetting amendment commending Reagan. This move, in turn, led the Democratic leader, Robert C. Byrd (W-Va.), to offer an amendment denouncing the administration for lax security precautions at the Marine post in Lebanon. And that set off the bitterest foreign policy debate the Senate has had since Reagan became president.

As word spread on Capitol Hill about the fireworks erupting on the floor, dozens of senators came running in to hear and join the argument. At one point, when Weicker and Steve Symms (R-Idaho) were trading shouted insults, the other senators sat watching them like spectators taking in a long volley at a tennis match.

For comic relief, the drama was interrupted at intervals by Sen. Robert J. Dole (R-Kan.), the floor manager of the debt ceiling bill, which was the pending business when the senators veered off onto foreign policy. Watching the day pass with no action on his bill, the exasperated Dole rose every half-hour or so to ask his colleagues, "Are we still on the debt limit?"

Baker was his usual composed self when he offered his resolution, which commended the president "for his swift and effective action in protecting the lives of Americans in Grenada."

Hearing this, Weicker stood up and charged not only that Reagan's invasion violated the law and the Constitution, but also that the administration was responsible for this month's coup in Grenada, and the murder by extreme leftists of Prime Minister Maurice Bishop. This is so, Weicker said, because the president and the the secretary of state both refused to see Bishop when he came to Washington in June.

"He had called himself a communist, but now he wanted to make a move toward the western democracies," said Weicker, who met with Bishop during the June visit. "Our government thought they'd stick it to Maurice Bishop, and now he's dead. And a lot of other people are dead, all because of that."

Symms, a strong conservative, now asked Weicker to yield the floor. Weicker declined. Symms starting shouting questions anyway: "Wasn't this Maurice Bishop a communist dictator?" Weicker shouted back even louder, complaining that "the far right . . . has called me a pinko and a communist."

At this point Baker took the floor and delivered a vigorous defense of Reagan's actions in Grenada. "How well I remember the humiliation of America in the streets of Tehran, and the seeming ineffectuality of America to do anything about it," he said. "America appeared reluctant, confused, and uncertain about its role . . . . But no longer!"

"No longer," Baker went on, in a reference to Cuba, "will we turn our back on a nation that insists on exporting war and revolution. Are we going to permit the implementation of the plans of these Soviet surrogates, or . . . will we take action?"

Despite the opening provided by Weicker, few Senate Democrats seemed inclined to challenge the Grenada invasion. Sen. Joseph R. Biden Jr. (D-Del.) said that after hearing a secret intelligence briefing on the matter, he had decided "the president was absolutely right." Sen. Daniel K. Inouye (D-Hawaii), who was at the same briefing, said he still needed to see further evidence before he could decide whether the invasion was necessary.

Instead, the Democrats took aim at security precautions at the Marine base in Beirut, where a suicide terrorist mission broke through guards and blew up the chief barracks, killing more than 200 Americans.

Byrd said he was "stunned into utter disbelief" when he read that Marine generals pronounced themselves totally satisfied with security arrangements before the blast. "Any man or woman, boy or girl in the United States can plainly see that these arrangements could not be totally satisfying to anyone," he said.

During this discussion, Majority Leader Baker, who has been a strong defender of Reagan's Lebanon policy in public comments, revealed that he sent a private letter to the president expressing "grave reservations" about the Marine deployment. He said he had urged the president to remove the Marines and "substitute some other international force."

As the debate moved along, it became clear that neither side was inclined to stop talking and vote on either the Baker or Byrd proposals. Dole rose and and suggested "it would be better if we could just bury the dead before we start all this commotion."

And then, as quickly as it had started, the great foreign policy debate was put to rest. Baker and Byrd both withdrew their amendments and turned the floor over to Dole. The Senate calmed itself and moved quietly back into its extended discussion of the national debt ceiling.