Republican and Democratic congressional leaders squared off on the national defense issue last night, then proceeded to agree on almost everything in the first of two debates sponsored by the League of Women Voters.

Consensus even reached the point at which Sen. Sam Nunn (Ga.) and House Majority Leader James C. Wright Jr. (Tex.), representing congressional Democrats, drew no fire from their Republican opponents as they charged that President Reagan had made a big mistake by trying to make a national defense issue of European willingness to buy natural gas from a Soviet pipeline.

"I disagree with the president," said Rep. Jack Edwards of Alabama, ranking Republican on the House appropriations subcommittee on defense, in discussing the pipeline. His Senate Republican ally on the panel, Armed Services Committee Chairman John G. Tower (Tex.) chose to stay out of that argument.

The four also agreed that the United States and Soviet Union should reduce nuclear weapons arsenals but said freezing them at current levels, as many groups around the world are advocating, is not the way to do it.

None of the four panelists would specify a figure when asked how much is enough for national defense. Wright, Nunn and Edwards also refused to blame the ballooning federal deficit on Reagan's military buildup, stating that the bigger culprit was the big tax reduction, which is giving the government far less money to pay its bills.

Nunn stood out among his colleagues in criticizing some of the specific assumptions underlying Reagan's five-year defense program but, in doing so, did not contest the size of the president's rearmament program.

"He who defends everything defends nothing," Nunn said in quoting a military maxim to make the point that Reagan is trying to do the impossible by telling the military services to be prepared to take on the Soviets throughout the world.

Nunn, a member of the Armed Services Committee, said military leaders are "puzzled" about what the president expects them to do.

Reagan has started "too many strategic programs at once," which will divert money from conventional forces that need modernizing, Nunn said.

Tower chose not to rebut Nunn point by point, settling for the assertion that "the United States needs the ability to respond to crises wherever they occur."

Wright said that the United States and Soviet Union have "enough weapons to kill each other three times over" and that it is time to cut military spending on both sides by 10 percent a year.