By a lopsided vote of 65 to 25, the Senate rejected last night the District of Columbia's proposal to build a $110 million convention center in the Mount Vernon Square area of downtown Washington.

The vote appeared to leave only a slim chance that the controversial project will remain alive, although its supporters continued to voice confidence.

The senate vote came on the District's $1.3 billion budget for fiscal year 1978, which started last Saturday. The measure now goes to a joint Senate-House conference committee.

The House voted last month to support the center, but a close vote on a related measure indicated little House enthusiasm for the project. That could weaken the bargaining position of the House conferees, who will be headed by Rep. William H. Natcher (D-Ky.), a supporter of the center.

Senate action on the city budget was delayed for almost three weeks by a fillbuster about natural gas and other legislative problems. With the filibuster broken Monday night, the Senate moved quickly to consider the budget.

For the first time in memory, the District currently has no legal power to spend money for operations, although the city is maintaining normal operations. Enactment of the budget would cure that situation.

The vote on the convention center was a clear victory for Sen. Patrick J. Leahy (D-Vt.), serving his first year as chiarman of the Senate District Appropriations Subcommittee. Leahy portrayed the center as a potential drain on city and federal taxpayers.

As approved by the Senate Appropriations Committee for floor action, the budget bill omitted $27 million in start-up costs for the project that is included in the House version of the measure.

Last night's action came on a motion by Sens. Lowell P. Weicker Jr. (R-Conn.) and Thomas F. Eagleton (D-Mo.) to add the $27 million to the Senate version.

Weicker said the Senate should support the District's elected government, which supports the project. He said such supports would give the city's limited home rule a chance to work.

Leahy insisted that the issue was "not a test of home rule since federal law continues to give Congress responsibility for reviewing the city's annual budgets and for setting the amount of the federal payment the city receives in lieu of taxes on government-owned property.

The Senate bill grants the District a federal payment of $276 million, or $24 million less than the $300 million maximum now set by law. The House version sets a federal payment of 296.4 million.

There was no clear pattern in last night's vote. Conservatives, for example, were on both sides of the issue, with Sen. Barry M. Goldwater (R-Ariz.) speaking in favor of the project and Sen. James Allen (D-Ala.) speaking against it.

"This is the brightest ray of fiscal responsibility I've seen in this chamber in many a day," Allen said.

Sen. Robert C. Byrd (D-W. Va.), the majority leader, opposed the project, while Sen. Howard Baker (R-Tenn.), the minority leader, supported it.

Of the four senators from Maryland and Virginia, only Sen. Charles McC. Mathias Jr. (R-Md.) supported restoration of the funds.

The project has been heavily lobbied by both sides. Washington business groups contend the center would restore vitality and economic life to a dilapidated section of downtown. Many citizen organizations oppose it as a drain on taxes. Some have sought a referendum on the project.

Clarence Arata, executive vice president of the Metropolitan Washington Board of Trade, and Joseph B. Danzansky, head of a Board of Trade task force on the center, watched the vote from the Senate gallery and emerged to voice disappointment and hope.