Secretary of Defense Caspar W. Weinberger talked rearmament to a conference of Republican governors here today, silencing those who have been urging President Reagan to shift some of his budget-cutting from their state capitols to the Pentagon.

With the exception of Illinois Gov. James R. Thompson, none of the 10 governors remaining at the annual meeting of GOP state executives challenged Weinberger's claim that "defense is unique" and security needs "cannot be put off like domestic problems."

Thompson said that he thought national security embraced "more than military hardware and assistance" and included a "federal contribution to ...a solid economy, jobs and a sound transportation system."

But while several of the others at the table had said previously they thought Reagan should include the Pentagon budget in his economy drive, none joined Thompson's dissent.

Weinberger told the Illinois governor the voters had given Reagan "a mandate" to improve military strength as a priority matter, and in a period of fiscal constraint "we have to have some cutbacks in domestic spending to be able to afford our defense."

"We will spend over $700 billion this year" in the federal budget, he added, "and less than $200 billion of that is for defense. So there ought to be a little left over for jobs, transportation and the rest."

Thompson asked if there was a "possibility of shifting just a few billion dollars" from defense to domestic programs, in order to ease federal-aid cutbacks to states hard-hit by the recession.

But Weinberger said any significant cuts in the Pentagon budget would mark a return to the erratic "feast and famine" defense policies which he said had enabled the Soviet Union to gain at the expense of the United States in the past decade.

Besides, Weinberger said, his experience as a budget director in the Nixon administration convinced him there was so much waste in the domestic budget that "I am not concerned that the reduction in programs" will harm the economy.

Other governors shifted the questioning to the issue of armed services manpower, allowing Weinberger to make the point that recent pay increases have "dramatically" increased the quality of military recruits while spurring both enlistments and re-enlistments.

Thompson later told reporters he "had expected that kind of response from Weinberger," adding that there was no sign of flexibility on the defense issue, from President Reagan or the defense secretary.