Britain's costly two-week-old national rail strike was called off today after the country's top union leadership in effect forced striking engineers to accept productivity rules demanded by state-owned British Rail.

The end of the walkout is a victory for the get-tough strategy and economic policies of Prime Minister Margaret Thatcher, who strongly supported British Rail's hard line in the dispute. Last week, the British Rail board said all strikers would be fired this Tuesday and the rail system closed unless the strike ended.

Thatcher has made tighter control of unions in the country's nationalized industries a major element of her efforts to revive Britain's economy. With the collapse of the rail strike, other public sector unions--including health workers, who are set to begin a three-day strike Monday--will find it harder to challenge government productivity demands.

The decision announced this morning by the Associated Society of Locomotive Engineers and Firemen became inevitable when the Trades Union Congress, Britain's equivalent of the AFL-CIO, agreed yesterday with British Rail's position on the introduction of flexible work schedules to improve worker efficiency.

"This was a battle which could not be won without the support and assistance" of the labor leadership in the congress, engineers' union leader Ray Buckton said. "This support is not forthcoming."

"The issue at stake was whether the organized trade union movement would allow the management of a nationalized industry to impose changes on its employes without their agreement," Buckton said.

The strike by the engineers' union's estimated 25,000 members was never popular with other British unions, especially the larger National Union of Railwaymen, which feared that a prolonged stoppage would lead to a permanent loss of rail routes and many jobs.

The Trades Union Congress' powerful Finance and General Purposes Committee stepped into the dispute after British Rail's threats to shut down the system. A marathon bargaining session that lasted from Friday until yesterday morning ended when the congress agreed to accept the introduction of "flexible rostering" at 71 of the country's 265 rail depots with negotiations to begin soon on bringing flexible work schedules to the rest of the depots. British Rail said it was satisfied.

In return for ending the strike, which officially is over at midnight, the engineers' union has had all dismissal notices lifted. In addition, the union will recall its annual membership conference to consider the strike's outcome. Union leaders pledged to endorse the concept of flexible rostering at that session.

The Thatcher government view is that long-term modernization of Britain's still-ailing economy requires abandonment of much of the power won by unions in the past century. The rules the engineers' union were refusing to modify, for instance, were set in 1919. Since then, rail use in Britain has declined substantially. Overmanning is considered a serious problem and the engineers' union is probably correct in anticipating that the productivity changes eventually will mean a loss of jobs.

British Rail, which lost, according to some reports, about $175 million as a result of the stoppage, has said it is preparing to make major new investment in the rail system to upgrade it and encourage greater passenger and freight traffic. A prolonged strike would have jeopardized those plans and also caused considerable losses to Britain's coal and steel industries, which rely heavily on the rail system.

The end of the rail strike by no means signals the end of Britain's labor troubles. The strike by a million workers in the nation's public health system is over government refusal to meet employes' demands for a 12 percent pay increase. At many hospitals only emergency services will be available.

The Press Association, Britain's domestic news agency, reported tonight that unemployment figures to be released Tuesday will show an increase to a record total of more than 3 million, or more than 13 percent of the work force.