By a vote of 13 to 7, the Senate Appropriations Committee dealt a serious setback yesterday to the District of Columbia's proposal to build a $110-million downtown convention center by refusing funds to start work on the project.

Sen. Lowell P. Weicker Jr. (R-Conn.), who led an attempt to provide $27 million of start-up funds, promised to fight for the money when the bill reaches the Senate floor, probably next week. He predicted his effort will have a fighting chance of success.

Only five of the Appropriations Committee's 25 members were present for yesterday's vote on the D.C. budget for the 1978 fiscal year, but a majority gave Sen. Patrick J. Leahy (D. Vt.) proxies to vote against the center. Leahy is chairman of the Appropriations subcommittee on the District.

City officials, backed by Weicker, have depicted the center as a potential generator of profitable new tax revenue.Leahy, who has depicted the center as a likely drain on city finances, prevailed last weeks when his subcommittee endorsed his position against the project.

The center project faces another test - and a probable victory - on Capitol Hill today, with the House sched- from A1> uled to act on its own version of the D.C. budget, which includes the $27 million for the center.

Rep. William H. Natcher (D. Ky.), chairman of the HOuse District Appropriations Subcommittee, is as strong a supporter of the center as Leahy is an opponent.

If the full House votes for the center and the Senate votes against it, the issue will have to be resolved by a joint conference committee.

D.C. City Council Chairman Sterling Tucker, who was in the audience at yesterday's Senate committee meeting and who tried without success to sway Sen. Henry Bellmon (R. Okla.) from his opposition vote, was disappointed but optimistic after the decision.

"I think the vote, while we didn't prevail, provides some strength by which we can go to the full Senate," he said.

At the District Building, officials voiced similar views when they heard of the vote.

"I expected that," said Council member Marion Barry (D-at large), chairman of the Council's finance and revenue committee. "The (District), subcommittee generally gets its way."

Barry's comment was more subdued than what he said last Friday, when the subcommittee rejected the project. At that time Barry called Leahy "a little rinky-dinky senator from a state like Vermont."

Yesterday Leahy angrily replied to Barry, calling the statement "childish" and "intemperate." Leahy also said the Barry statement was published in Vermont newspapers, leading some constitutents to send letters urging Leahy to stand fast against the District project.

Weicker, in seeking restoration of the money, said the principle of home rule for the District is at stake.

"I think the mayor and Council ought to have the dominant say," Weicker said, adding later: "It's not a question that the District has to cut the apron strings (from the federal government). We've got to cut the apron strings up here on (Capitol) Hill."

The convention center is proposed for a site in the dilapidated Mount Vernon Square area, north of downtown. It would occupy three blcoks on the south side of New York avenue between 9th and 11th Streets NW.

Under the proposed financing plan, the District would ultimately borrow the entire $110 million construction cost from the U.S. Treasury, repaying the money with interest over a period of years. Treasury borrowing is the only way the city now has of financing construction projects, although it hopes within a few years to switch to the issuance of municipal bonds, like most cities do.

The center project has stirred the biggest controversy in the pending budget, which totals $1.3 billion in its Senate version and $1.4 billion in the House version.

Another controversy is emerging over the size of the federal payment, the sum the government pays that makes up the difference between District revenues and the total amount of the budget.

Federal law authorizes a payment up to $300 million. The pending House version provides $296.4 million. The version approved yesterday by the Senate committee provides $276 million.

Sen. Charles McC. Mathias Jr. (R. Md.), the District Appropriations sub-committee's top Republican, sought to raise the Senate outlay to $296.2 million, saying the proposed cut was an unwarranted penalty for the city. The Mathias motion was rejected by a tie vote.

There was no discussion of the sub-committee's recommendation that $500,000 be appropriated to continue operations of the city's 36 advisory neighborhood commissions, an experiment in grass roots government. The House version knocked the money out entirely.

In yesterday's vote on the convention, center, Leahy won support from the Appropriations Committee's top leadership, the ailing Chairman John L. McClellan (D. Ark.), who voted by proxy, and Acting Chairman Warren G. Magnuson (D. Wash.), who was present. Senate Majority Leader Robert C. Byrd (D. W. Va) also voted against the project by proxy.