The Senate voted 91 to 4 yesterday to relax nearly a century of government regulation of the nation's railroads after a last-minute compromise averted a fight over coal-hauling rates that could have sidetracked the whole bill.

Although it does not go as far toward total deregulation as the Carter administration or the industry wanted, the legislation would give individual railroads more flexibility in setting rates -- which rail companies say they need to survive.

It also would streamline some regulatory procedures, making it easier for railroads to merge and reorganize, and would resolve a longstanding dispute within the industry over joint rates for a single shipment.

The measure is the second of three major transporation bills to come before Congress in recent years. Two years ago Congress eased government controls over airlines, and a bill to relax regulation of the trucking industry is expected to come before the Senate after the Easter recess.

In another reflection of the antiregulation mood of Congress, legislation to phase out ceilings on bank interest rates also won final passage by Congress last week.

Hearings on a rail deregulation bill that is broader than the Senate version will begin today before a House Commerce subcommittee.

The way was cleared in the Senate for the rail bill when, 20 minutes before the debate began, Sens. Howard W. Cannon (D-Nev.) and Russell B. Long (D-La.) reached a compromise over limitations on rail charges for transporting coal -- a dispute that had pitted the nation's electrical utilities against the railroads in a lobbying scrap that clouded prospects for the deregulation bill.

Long, representing a Deep South state that pays dearly for rail-borne coal from Wyoming and Montana, had wanted to make it difficult for railroads to raise their coal-hauling rates above a trigger point established by the Interstate Commerce Commission when no other means of transportation is available.

But the railroads claimed that an amendment proposed by Long would burden them with more, rather than less, regulation and vowed to try to kill the bill if Long prevailed.

Vote counts over the last few days, however, indicated that Long wouldn't succeed, even though a number of consumer groups had climbed aboard the utilities' lobbying bandwagon.

So Long compromised, forsaking his earlier demand that railroads have to prove their need for rate increases above the ICC trigger point but winning on some other points.

In deciding whether to investigate a rate increase, the ICC would be required by the compromise to consider factors such as the impact of the increase on national energy policy, which, as Long has argued, requires even such oil-rich states as Louisiana to convert to coal for electricity generation. In deciding whether a rate is reasonable, it also would have to consider whether one commodity, such as coal, provides a disproportionate share of a railroad's revenues.

The ICC also would have to issue a detailed explanation of its reasons for refusing to investigate a rate increase, which Long said would enhance congressional monitoring of the commission's rate reviews.

Long conceded afterward that he was dealing from a position of weakness, saying, "If I felt confident that I had 10 votes . . . I wouldn't have compromised."

Although Cannon clearly had the votes to win, according to a source on the Senate Commerce Committee, he was moved to compromise for a number of reasons, including "that old saying that, even when you beat Senator Long, you lose."

Both the railroads and utilities expressed mixed feelings about the compromise and indicated they would resume their fight in the House."I think it's a wash," said Edward Milne, lobbyist for the Edison Electric Institute. "Adding all the pluses and minuses, I think we got more pluses," said Edward T. Breathitt Jr., vice president of the Southern Railway.

The Carter administration hailed the bill's passage, saying it would "free the railroads from unnecessary and harmful ICC regulation" while providing protection for shippers.