The billion-dollar improvement program for the Northeast rail corridor, intended to make trains travel between Washington and Boston safer and swifter, has left waste, confusion and danger in its tracks, according to a General Accounting Office study released yesterday.

The report is highly critical of every aspect of the project, which is entering its fourth year. It cites hundreds of millions of dollars in costoverruns above the $1.75 billion initially appropriated for the program. It details poor workmanship and concludes that some track work already done likely will have to be redone. It presents a record of crises and conflicts among the three agencies chiefly responsible for the project.

"Because of the high potential costs involved, Congress should decide what improvements are desirable and what additional funding, if any, should be provided," the report said. "Alternatives range from terminating the project to providing over $5 billion to make capital improvements to achieve the (original) objective."

Sen. Charles Mathias (R-Md.), who had asked for the report, announced that the transportation subcommittee would hold hearings in two weeks to let the Federal Railway Administration-which has overall responsibility for the Northeast Corridor plan-tell its side. Despite the overwhelmingly negative conclusions of the GAO study, the senator sounded an optimistic note in remarks to reporters.

"Like any report card, you ough to view it as a point for departure and improvement," he said. "It's early enough in the game that we can correct the situation. We hope this study gives us the opportunity for the stitch in time."

GAO investigators were less sanguine. They said the project has been managed poorly from the start. The three agencies that have had varying degrees of respomsibility for it haven't worked well together, the GAO said.

The agencies are the Federal Railway Administration, which have overall responsibility; Amtrak, which owns the corridor and operates the trains; and Deleuw, Cather/Parsons Associates, which is the project's architect/engineer.

The report recommended that Amtrak take over primary authority for the project. "The Federal Railroad Administration has not made overall decisions and has not defined the project or what the roles of the major participants are," said the report. GAO sees no overriding reasons for keeping the FRA directly involved in the project."

But administration officials contended yesterday that it won't be necessary to change command of the project. They said steps already had been taken to correct many of the report's criticisms as a result of a Department of Transportation "redirection study" last year.

"We feel that most of the issues raised (in the report) have been resolved," said Louis S. Thompson, the FRA's corridor project director. Among the improvement Thompson cited was a recent "memo of understanding" agreed to by the FRA and Amtrak to settle conflicts and define roles.

Transportation Secretary Brock Adams already has asked Congress to approve an additional $654 million for the project this year. In submitting the request in January, Adams disclosed the project's scope has been reduced and its scheduled completion time extended to the end of 1983 (1984 in the Boston area).

When first outlined in 1976, the project called for regularly scheduled, dependable train service and a running time of 2 hours and 40 minutes from Washington to New York, adn 3 hours and 40 minutes between Boston and New York by 1981. It was the largest and most complicated rail improvement project ever tried.

As a result of the reductions in scope since, the GAO claimed there will be increases in maintenance costs, and reductions in passenger comfort, on-time reliability and safety.

For example, the report noted that fences on bridges which pass over the railroad havd been shortened, leaving many under 8-foot-high project minimum. The standard had been developed to shield trains from thrown objects that othersiwe could cause damage and injuries.

The report also said that track work often was completed in the past two years before engineering plans were prepared. As a result, some of the work probably will have to be redone, the GAO said.

In addition, government investigators cited numerous instances of delays, costly changes in plans and poor workmanship. In one case, Amtrack was said to have installed 50,000 concrete ties that may all be below specifications.