Senate Armed Services Committee Chairman Sam Nunn (D-Ga.) charged yesterday that President Reagan's plan to give U.S. protection to Kuwaiti tankers "poses substantial risks" of a violent confrontation with Iran that could strengthen the hand of the Soviet Union in the Persian Gulf region.

In a highly critical assessment of the planned reflagging and escort operation, submitted on the eve of a meeting between Reagan and congressional leaders on the issue, Nunn renewed his call for a delay of the operation but cautioned against any "restrictive legislative actions" to block it.

American interests "would be best served by cooperation . . . not by confrontation," he said.

Senate Majority Leader Robert C. Byrd (D-W.Va.) and House Speaker Jim Wright (D-Tex.) are expected to urge Reagan to delay the operation when they meet with him this morning. They also are considering legislative action if he refuses, although it is considered unlikely to have the effect of halting the operation if, as expected, the administration goes ahead with the reflagging and escort protection of 11 Kuwaiti tankers next month.

"The votes are there for a delay," said House Democratic Whip Tony Coelho (D-Calif.). "But legislatively there is no way to stop it if the president wants to move along. It would be a purely political statement."

Elaborating on Coelho's remarks, a senior Democratic aide said that, while Congress could muster a simple majority for a delay, it could not produce the two-thirds vote necessary to override a veto.

In remarks at a luncheon meeting with reporters, Byrd said it would be difficult to reverse a reflagging decision because lawmakers do not want to appear to be "jerking the rug out from under the president" and "further undermining the credibility" of the United States in the gulf region.

But he did not rule out congressional action of some kind, and the Senate Foreign Relations Committee is expected today to consider legislation that would delay the reflagging for a year.

Legislation to require a delay is also under consideration in the House.

Nunn's 13-page report to Byrd was one of the strongest, most sweeping denunciations of the reflagging proposal so far and was said to reflect many of the points that the congressional leaders will make to Reagan today.

It asserted that the United States and its allies have a "vital strategic interest" in the Persian Gulf but said these interests are "not being substantially challenged at this time" and questioned the administration's claim that U.S. action is necessary to prevent the Soviets from extending their influence in the gulf.

But it also warned that the reflagging operation "poses substantial risks of a U.S.-Iranian confrontation which could escalate violently and unpredictably," paving the way for the Soviets to "gain substantial strategic advantages in terms of closer Soviet-Iranian relations."

It expressed strong doubt that the American people or Congress would offer "sustained support" for military action if the United States is involved in hostilities in the gulf.

The report contended that an expanded U.S. military role would not help end Iran-Iraq hostilities and raised the specter of the U.S. debacle in Lebanon in questioning the strategic rationale behind the reflagging.

"The administration proposes, just as it did in Lebanon, to place U.S. military forces in harm's way without formulating and articulating clear and attainable military objectives," the report said.

As alternatives to the reflagging, Nunn suggested that the administration continue to pursue an end of the Iran-Iraq war through the U.N. Security Council. He also advocated other moves to pressure Iran to accept a negotiated settlement, including efforts by the United States, the Soviet Union and other major powers to curtail arms shipments.

Nunn also proposed a conference of major Persian Gulf oil exporters and importers, alternative arrangements with Kuwait such as lease of American vessels and a bipartisan congressional expression of support for U.S. interests in the gulf.