A new study of Amtrak's performance and future potential has concluded that the taxpayer-subsidized passenger service was dommed to fail from the start and has "simply been too expensive" in exchange for little contributions to the overall society or transportation goals.

Prepared for the National Transportation Policy Study Commission, established by Congress in 1976, the report says that only in a few densely populated corridors does Amtrak approcah a "reasonable" return for the taxpayers' investment.

"The only major difference between past and present rail operations since the coming of Amtrak is that subsidies are now covered by the government, rather than by the privately-owned railroads . . . a complete network of intercity rail passenger services cannot be operated in the U.S. on a for-profit basis," said Northeastern University Prof. Frank Mulvey in the new study.

The transportation commission is charged with making recommendations on policy through the year 2000, it is composed of 12 members of Congress and seven public members named by the president. The initial draft of the Mulvey report does not represent findings of the commission but will be used to solicit comments for inclusion in final recommendations.

In general, Mulvey found Amtrak's contribution toward transportation goals to be negligible. In addition, the national rail network has not had a significant impact on environmental protection or energy conservation, since only a small number of Americans ride trains.

Amtrak, formally the National Rail Passenger Corp., was established by Congress in 1977 and given the task of maintaining a skeletal network of intercity train services, following massive losses on passenger operations by private companies. Annual subsidies to Amtrak are now running at about $500 million.

Not counting automobile travel, which accounts for the bulk of Americans' intercity travels, Amtrak has only 2 percent of common carrier business compared with about 80 percent for airlines and 15 percent for buses. In the Boston-Washington corridor, however, Amtrak estimates its share of market at about 16 percent (again, not counting cars).

Even if every available seat on every Amtrak train could be filled, the service would continue to have an annual loss of up to $100 million, Mulvey said. He also discounted comparisons with Japanese and European rail operations, stating that the foreign nations have shorter travel times between major cities, higher air fares, higher gasoline prices, less developed highways and a trend toward more auto use as incomes increase.

He recommended that Congress abandon the goal of a national system and conceded that rural and less populated areas would lose all service under such a plan. "It is also true that small towns are without subways, international airports or . . . hospitals that can perform heart transplant operations," he stated.