President Reagan said yesterday that "vital weapons systems, either conventional or strategic, must not be touched, period" in any budget compromise with Senate Republicans, although he left open the possibility of some accommodation on Pentagon spending.

Rep. George E. Brown Jr. (D-Calif.), in a reply to the president's Saturday radio address, said Reagan's budget is "a complete hoax."

Reagan took a hard line against a slowdown in defense spending a day after he agreed to start negotiations with Senate Republicans aimed at a budget compromise.

"Don't believe the drumbeat of propaganda that blames defense spending for government living beyond its means," Reagan said. "Our bipartisan effort to rebuild America's defenses only began three years ago, after more than a decade of neglect, while the Soviets surged ahead with the greatest military buildup in history, adding countries to their empire with the ease of a thief plucking apples off a tree.

"The deficit can and will be brought down, but not by raising taxes, which would just torpedo growth and make the deficit worse, or by gambling with America's security, when the Soviet Union is every bit as aggressive, expansionist and dangerous as before," he said.

Defense spending has been one of two major differences between Reagan and Senate Republicans. The Republican-controlled Senate Budget Committee voted for a budget with no growth in defense above inflation for next year, far less than Reagan sought.

The other difference is Social Security cost-of-living adjustments, which the Senate panel voted to freeze for one year; Reagan opposes any delay.

Reagan said yesterday, "I'm confident that we're coming closer to a meeting of the minds" with Senate Republicans on a deficit compromise.

He said they are in agreement that "uncontrolled spending poses a threat to our expansion," but Reagan added that on Friday, "I made clear that in further reduction in defense, vital weapons systems, either conventional or strategic, must not be touched, period."

Reagan did not say what defense reductions, if any, he would accept. Officials have said previously that the Senate Budget Committee plan would affect weapons systems.

Brown, in the Democratic response, said Reagan once again was laying blame for deficits on "an uncooperative Congress or prior administrations. Domestic spending is bloated and defense has been starved . . . . The flaw in this line is that it is a complete hoax."

"If every line of the president's budget were enacted, we would continue down the road to national bankruptcy that we have followed the last four years with $200 billion annual deficits indefinitely," Brown said.

"Domestic programs have been cut to the bone in each of the last four years," he said. "Our defense buildup has been the greatest in history over the past six years, beginning under President Carter, while the Soviets maintained a slow and steady growth of around 2 percent until recently, according to the CIA."

Brown said "the president's hoax is so obvious" that the Republican-controlled Senate Budget Committee rejected his spending plan, 17 to 4, and came up with its own, producing deeper cuts in the deficit. He predicted the House will do the same.

Reagan's proposed budget for fiscal year 1986 includes a $180 billion deficit, but yesterday he said Congress was at fault for what he called an "ocean of red ink."

"I'm sorry to tell you that many of those you elected to keep watch over government's expenditures and to keep your taxes down have been hard at work since we submitted our budget doing something else," he said. "They've been promising special interest groups not to save your tax dollars, but their spending programs."

Reagan said yesterday, as he did in Thursday night's news conference, that his defense budget this year is "less than my predecessor predicted he would spend in his last budget."

However, Carter's last defense projections were based on estimates of the high inflation then prevailing.

Congressional studies have shown that Reagan's buildup is about 10 percent higher than Pentagon spending would have been under Carter's policies between 1981 and 1984 if inflation levels are taken into account.

Brown also criticized Reagan's Strategic Defense Initiative or "Star Wars" missile defense system, saying it cannot fulfill the promise of protecting Americans from incoming Soviet nuclear missiles.

Reagan's "dream of creating a shield over the United States to protect against nuclear attack is compelling, but so is the idea of finding the fountain of youth or turning lead into gold as the ancient alchemists sought to do. All these dreams are equally improbable," he said.