Plans to extend the west front of the Capitol were rejected yesterday by the Senate Legislative Appropriations Subcommittee, probably killing the controversial project at least for this session of Congress.

The action was taken with the apparent blessing of the Senate's top leadership, which had joined with House leaders in April to endorse the $55-million project. Despite their past support for the project the Senate leaders never have strongly pushed for it.

Yesterday's decision was a victory for preservation groups, spearheaded by the American Institute of Architects, which protested that the extension plans would cover over the last remaining visible section of the original 177-year-old Capitol building.

Extension plans have sparked congressional controversies for 12 years. The House repeatedly has approved them - most recently earlier this week - and the Senate has blocked them.

The House, with a membership more than four times the size of the Senate, always has felt cramped in the space allotted to it in the Capitol. Many members say they need more space to meet constituents.

Enthusiasm for major construction projects dates from the days of Speaker Sam Rayburn (D-Tex), who died in 1961. His successors have continued to support expansion plans.

Sen. Walter Huddleston (D-Ky.), the subcommittee's chairman, told a news conference yesterday that his panel will urge the Senate to direct Capitol Architect George M. White to prepare plans for cheaper ways to restore the deteriorating sandstone facade.

While taking note of both the preservationists' views and the Capitol's structural problems, Huddleston did not make it clear which issue was more persuasive.

Huddleston said seven proposals have been advanced by various people, ranging in cost from $3 million to $45 million.

The west front is the Capitol facade that faces the Mall amd downtown Washington. An extension of the east front, facing the Capitol Plaza, and the Supreme Court building, was completed in 1961.

Yesterday's annoumcement does not kill the west from project outright, but leaves its future very much in doubt.

Both chambers of Congress must agree on a single course of action - either the House's decision to proceed with the project, or the Senate subcommittee's proposal to sidetrack it.

Sen. Richard S. Schweiker (R-Pa.), ranking minority member of Huddleston's subcommittee, said the closeness of this week's House vote - 212 to 204 - strengthens the Senate's hand.

However, both Huddleston and Schweiker said they would reconsider the extension proposal if the final estimated cost of preservation came anywhere close to the $55 million cost of the extension.

"We don't believe that's going to be the case," Huddleston said.

Past proposals for the extension have called for adding up to 44 feet to the facade of the center portion of the building. That would have required rebuilding the broad terraces designed in the late 1800s by Frederick Law Olmstead, a famed landscape architect.

White, who became the presidentially-appointed Architect of the Capitol in 1971, devised the current plan, where would push the facade westward 22 feet. The Olmsted terraces would remain untouched. The existing sandstone exterior facade of the building would become an interior wall.

White said the new facade, which would be built of marble, would be designed to withstand the outward pressure of the Capitol building's heavy weight, which now pushes against the relatively fragile sandstone.

Backers of the extension project have stressed the potential danger of a possible ultimate collapse of the Capitol's present facade, while opponents of the project have minimized the potential problem.

White's plan was approved April 7 by the Commission on the Extension of the Capitol, a body created in 1955 that includes both House Speaker Thomas P. O'Neill (D-Mass.) and Vice President Mondale, who is sthe presiding officer of the Senate. Its other members are the majority and minority leaders of both chambers.

Huddleston and Schweiker said they talked with Sen. Robert C. Byrd (D-W. Va.), the majority leader, and Sen. Howard Baker (R-Tenn.), the minority leader, and - according to Huddleston - found that "they would be strongly influenced by our recommendation . . . We got the impression they would rely on the recommendations of (our) subcommittee."

Actually, the Senate subcommittee has not met to decide the issue, although members have discussed it informally. "I got tired of answering questions form individual reporters" and decided to call a news conference to announce the suncommittee's position.

Huddleston said the subcommittee will call upon White to conduct a new study of how space is used in the Capitol. He said the intent of the study would be to see whether some congressional functions could be shifted to the House and Senate office buildings form the Capitol itself, freeing space for more vital congressional activities.

Rep. Samuel S. Srattion (D-N-Y), who led opposition to the west front project in the House this week, said by telephone from his district office in Schenectady that the Senate subcommittee's proposals make a lot of sense.

"This, of course, doesn't mean that the battle (against the west front extension) is won," he said.

Other members of Congress active in the dispute could not be reached. White, the Capitol architect, is vacationing in Canada.