Leaders of both parties in the Senate agreed yesterday that President Reagan probably has the support of Congress, at least as it returns today from its five-week summer recess, to keep the U.S. Marine peace-keeping force in Lebanon.

But there was disagreement over whether the War Powers Act of 1973 requires him to get congressional authorization for the deployment, and some restiveness over the extent of long-term U.S. policy-making, or lack of it, for Lebanon and the rest of the Middle East.

In separate appearances on television interview programs, Senate Majority Leader Howard H. Baker Jr. (R-Tenn.), Minority Leader Robert C. Byrd (D-W.Va.) and Sen. Richard G. Lugar (R-Ind.), a member of the Foreign Relations Committee, said they thought that Congress, if asked, would authorize the marines to stay.

Baker conditioned his response with the word "probably," explaining that, while "I'm sure at this moment . . . I'm not sure what the situation will be in 10 days or two weeks."

Dividing the lawmakers is the issue of whether the recent shelling deaths of four marines requires Reagan to seek congressional approval for the marines to remain in Lebanon under provisions of the War Powers Act requiring such action when U.S. forces are involved in "hostilities."

Reagan, pointing to a different provision of the act, contends that he isn't required to seek congressional authorization. But the law allows Congress to act in the absence of a presidential request, and Sen. Charles McC. Mathias Jr. (R-Md.) said Saturday he will introduce a War Powers Act resolution today to authorize the marines to stay.

Byrd, appearing on "This Week With David Brinkley" (ABC, WJLA), said he thought it was "obvious the situation requires the application of the War Powers Act," under which Congress would have 60 to 90 days to continue, terminate or modify the deployment.

Lugar, appearing on the same program, indicated that he thought the law shouldn't be triggered yet, but said the United States is approaching a "crossroads" on the issue.

"When they're shooting at you, you're very near a combat situation," said Senate Finance Committee Chairman Robert J. Dole (R-Kan.) on "Meet the Press" (NBC, WRC). But Dole said the law is "fuzzy" on some points and suggested the possibility of a separate congressional resolution, which some lawmakers are exploring as a way of avoiding a confrontation with Reagan over applicability of the 1973 law.

Baker, appearing on "Face the Nation" (CBS, WDVM), skirted the War Powers Act issue by complaining that Congress shouldn't let a "legalism" get in the way of the basic question of what the U.S. role in Lebanon should be.

"The real question is what are we going to do in the long term in the Middle East, and that policy has not yet been fully determined," he said. Baker suggested that the United States faces an "either/or" situation in which it must decide whether to "cut loose from that area or resolve to do whatever is necessary to preserve the security and sovereignty of the state of Lebanon."

Lugar expressed concern that any time limits on the deployment of the marines would undermine the effectiveness of U.S. policy in the region. But Byrd said a congressional authorization would assure a "coalescence of will" behind the U.S. military presence.

The Senate leaders' assessment that Congress will keep the marines in Lebanon, at least for now, comes in the face of reports from many rank-and-file members that their constituents are uneasy over the continuing presence of U.S. forces there. But many members also indicate ambivalence, if not reluctance, toward congressional involvement in the situation.

Senate and House foreign policy committees will be briefed on Lebanon Tuesday by the administration. And the congressional Republican leadership also is expected on Tuesday to discuss the issue with Reagan. A GOP leadership aide said no decision has been made on how to handle the War Powers Act controversy.