Railroad labor's top leaders voted unanimously late yesterday to support Alleghany Corp.'s proposal to acquire Consolidated Rail Corp. from the federal government.

The endorsement -- which depends on "refinements" in the tentative agreement between the unions and Alleghany and an immediate return of Conrail's 35,000 unionized employes to industry wage rates -- was a significant, if not fatal, blow to the bid of Norfolk Southern Corp. to acquire Conrail and set up a giant rail complex.

Other bidders hoping for labor's endorsement were Conrail's management and a group of investors headed by hotel executive J. W. (Bill) Marriott Jr.

Transportation Secretary Elizabeth Hanford Dole has been waiting for labor to state its preference before announcing her own choice and recommending it to Congress, which she is expected to do later this week. "I have been interested in the views of labor since the beginning of this process and will take their recommendation into consideration," she said last night.

Norfolk Southern was widely regarded as the front runner with the department, assuming it could resolve potential antitrust problems, but a department official said last week that "a unanimous labor veto" could seriously damage Norfolk Southern's prospects.

Labor's action is not a veto of anyone, but O. M. Berge, chairman of the 19-member Railway Labor Executives Association and president of the Brotherhood of Maintenance of Way Employes, said "the important thing is that railway labor is unanimous."

Richard I. Kilroy, international president of the Brotherhood of Railway, Airline & Steamship Clerks (BRAC), said the decision was reached "through a process of elimination . . . If you can't get your choice, you go to the best that you have."

Kilroy had announced Friday that his union supported a public sale proposed by Conrail's management, a position strongly opposed by Dole.

Berge and Kilroy said they are prepared to support Alleghany. "I would think we would fight it in Congress if Dole chose any" bidder other than Alleghany, Kilroy said.

Alleghany, a New York holding company with a long history of railroad involvement, opened the bidding last April 10 and was one of three finalists chosen by Dole. Conrail management's proposal for a public sale forced its way into the process, but never attained Dole's backing.

Berge said "public offerings on the table are extremely risky from my point of view. I feel better with somebody who can come along and write a check." Critics have characterized the Conrail management's public offering as a leveraged buyout in disguise, with the assets of the railroad pledged as security against the future. Dole wants a financially strong railroad, one that would be unlikely to return to federal receivership. Norfolk Southern's financial strength is its greatest asset in the contest.

Alleghany has offered $1.2 billion in cash for the railroad, much of which it raised in a January 1984 sale of Investor Diversified Services to American Express. It has a problem on Capitol Hill, particularly with Rep. James J. Florio (D-N.J.), chairman of a House Energy and Commerce subcommittee that must handle Conrail sale legislation.

Alleghany officers were at one time on the board of the Penn Central railroad, which went bankrupt before becoming a predecessor of Conrail in 1976. Kilroy said that persuading Capitol Hill to accept Alleghany "will be difficult" because of the perception by some that "Alleghany was a pall bearer at the funeral of the Penn Central."

John J. Burns, president of Alleghany, said last night "we're very happy with the news and are pleased with the support of labor." He said that the concerns about Alleghany's past are concerns "I've never really understood."

Norfolk Southern, Marriott and Conrail declined comment.

In addition to cash, Alleghany has offered Conrail employes 35 percent ownership of Alleghany Corp., three seats on the Conrail board and one seat on the Alleghany board, Kilroy said.

Alleghany also agreed -- as did all the prospective buyers -- to forego tax credits Conrail owns and to restore Conrail's employes to full industry wages. They are working for 12 percent less under an agreement they reached with the railroad to help restore it to prosperity. After absorbing $7 billion in taxpayer subsidies since its formation in 1976, Conrail is expected to show a $500 million profit for 1984.