The House Appropriations Chairman and the nation's top military officer clashed yesterday on the question of whether Congress had provided too little, too late for national defense.

During your stewardship of the last three years, Chairman George H. Mahon (D-Tex.) told Gen. George S. Brown, chairman of the Joint Chiefs of Staff, Congress has appropriated $330.7 billion for national defense.

"It's rather discouraging to learn from you that this did not buy us the security we need and that the risk has increased," said Mahon in response to Brown's written report to the defense appropriations subcommittee.

Brown, who leaves office this summer, said in his farewell report that "in nearly every area of military strength there has been a relative decline over the years in relation to the Soviet Union." He added that "the military risk to the nation will increase."

Why, asked Mahon, did the $330.7 billion fail to buy the nation more security? Was the relative decline "inevitable?"

"In hindsihgt," replied Brown, "I don't think I would have said it is inevitable" that the U.S. military position would slip relatively. However, Brown continued, Congress failed to appropriate all the money the Pentagon requested - cutting the budgets by an average of $5.5 billion for each of the last five years.

Pressed by Mahon to name one single program that Congress killed which the military considered vital, "something big or outstanding," Brown said he could not think of one offhand.

"I'd have to check the specifics," Brown said. "I would have to back and look."

"You are aware," Mahon continued in his lecture to Brown, "that most of the reductions have been postponements because the Defense Department was not ready for the funds."

"Would you have been optimistic" about the future U.S.-Soviet military relationship if Congress had provided all the funds the Pentagon requested Mahon asked.

"No sir," Brown replied.

At this point, Defense Secretary Harold Brown, who was sitting beside the general at the subcommittee witness table, broke into the exchange to say that "the key to this matter" is that "the Soviet buildup has been faster than anticipated."

Continuing his challenge of Gen. Brown, Mahon said, "My colleagues keep asking, 'How do you let them get away with so much waste?' The average American would feel that $330 billion not only would enable us to avoid disaster but to keep pace with the Soviet Union."

Gen. Brown replied that "the rather staggering amount of money" Congress has appropriated to the Pentagon in the past "is acknowledged." But, continued the nation's top-ranking general, it "should come as no surprise" that the United States is being pressed militarily because "the other fellow is spending more."

Rep. Joseph Addabbo (D-N.Y.) warned Gen. Brown and Secretary Brown that "we could spend ourselves into oblivion" by using the nation's resources to combat an outside threat and thereby destroying the country from within in the process.

Switching to the topic of the B1 bomber, which President Carter canceled, Addabbo asked Gen. Brown whether he favored spending $462 million to build two additional test models, even though the bomber's future production has been canceled.

"I think the program is dead and therefore I would not spend half a billion dollars" to buy two additional test models of the B1 bomber, Gen. Brown replied.

"I do feel the B52 is not going to last forever," Gen. Brown said.

The Pentagon is studying bomber replacements for the B52 and B1 but sees no need to rush into development.