The House voted yesterday to kill funding for any additional MX missiles next year and to cap the number of missiles the United States will deploy at 40, fewer than half the number President Reagan sought for a program he termed the centerpiece of his defense buildup.

The action sets up a confrontation with the Senate, which three weeks ago voted funds for 12 new missiles and expressed the sense of the Senate that no more than 50 be deployed. The White House, which has long pressed for 100 missiles, was forced to back the Senate action to avoid a complete MX defeat.

Asked at his news conference last night if he would accept 40 MXs rather than 50 "if Congress gave you an extra $200 million for the Midgetman" program, Reagan said:

" . . . We'd have to look at that very seriously . . . to see whether there was an advantage." The Midgetman missile is a smaller, more mobile version of the MX.

Reagan also said that "the debates that are going on about the MX . . . are a lot of wasted rhetoric, and we ought to get on with it."

The House action occurred on an amendment to the 1986 defense authorization bill, which as approved by the conservative-dominated Armed Services Committee would have provided $2.1 billion for 21 additional MX missiles.

The amendment was adopted by voice vote, but in an earlier test vote the House approved the 40-missile cap 233 to 184 with 201 Democrats and 32 Republicans voting in favor.

The House voted down, 234 to 182, a GOP-sponsored amendment similar to that passed by the Senate. The House also voted down, 230 to 185, an amendment by Rep. Charles E. Bennett (D-Fla.) that would have terminated the MX program.

Opponents of the MX said yesterday that the House votes, and the earlier Senate action, indicated a weariness with the nuclear missile and a desire to end further debate over how many of the 10-warhead weapons to deploy in western silos that many lawmakers consider vulnerable to Soviet attack.

Congress has already approved 42 missiles -- two of which could be used for testing -- although the first missile will not be deployed until December 1986. Just last March Congress agreed to release $1.5 billion for 21 of those missiles. At that time the House voted 219 to 213 in favor of releasing the money.

The House also voted to cut $10 billion from the $302.6 billion authorized by the Armed Services Committee for total defense spending in fiscal 1986.

The authorization brought up yesterday is only part of that total defense number; personnel costs and a military construction bill also are factored into it.

The cut, approved 301 to 115, freezes defense spending next year at this year's level, with no provision for inflation. The House earlier endorsed a defense freeze in its 1986 budget resolution.

The Reagan administration had initially requested $322.2 billion for defense but, in an agreement with the Senate, had gone along with a $302.6 billion defense bill approved by the Senate two weeks ago.

The MX amendment approved by the House yesterday was one of several amendments that Armed Services Committee Chairman Les Aspin (D-Wis.) and Democratic leaders put together over the last few months to "reorder" defense priorities away from nuclear and more toward conventional forces.

It is also designed to help the Democrats shake the image that they are antidefense, a charge repeatedly leveled at them by the Republicans in recent elections.

While proposing a reduction in funds for the MX, for antisatellite weapons, and Reagan's controversial space-based nuclear defense system commonly called "Star Wars," the Democratic package would increase funding for the Midgetman missile and for conventional forces.

All of those amendments are expected to be the focus of sharp debate this week.

The biggest fight is expected over Star Wars, a research program of space-based lasers and other high-technology devices that the administration hopes would eventually allow the United States to destroy attacking missiles.

Reagan requested $3.7 billion for Star Wars. The Armed Services Committee reduced that to $2.5 billion. The Democratic package amendment, offered by Rep. Norman D. Dicks (D-Wash.), would cut it further, to $2.1 billion, and limit funding for four programs in Star Wars that might violate the 1972 Antiballistic Missile Treaty.

Another fight is expected to occur today over the committee's decision to give Reagan his full request of $124.5 million for chemical weapons. Congress has eliminated funds for such weapons in the past.