The General Accounting Office last fall withheld a report suggesting that the Army Corps of Engineers misrepresented the costs and benefits of the most expensive waterway project it ever had undertaken.

The withholding of the report, apparently at the behest of Sen. John C. Stennis (D-Miss.), a staunch supporter of the project, is revealed in documents related to an upcoming trial in federal court in Mississippi.

Pretrial activity in the little-publicized suit aimed at stopping construction of the $1.8 billion Tennessee-Tombigbee Waterway has uncovered an array of damning information from files of corps.

Among the items:

In the summer of 1975, corps officials and planners debated at length over the sobering realization that the project probably was being built illegally - almost twice as wide as Congress had authorized.

After Stennis called Comptroller General Elmer B. Staats, the GAO last year shelved the study that found the corps had inaccurately calculated the benefits and costs of the canal.

Economic calculations the corps used to justify the project were geared to the unauthorized width of 300 feet the engineers were planning for the canal. At the authorized width of 170 feet, the project could not be economically justified. By law, benefits must exceed costs before a water resources project can be built.

Benefit ratios were drawn up without considering that at least $280 million more would be needed, but which Congress had not authorized, for widening and straightening the Tombigbee River in Alabama, to which the canal would be tied.

Cost overruns up to $400 million were being hidden, to lessen the "emotional impact" on Congress, as Army investigators phrased it. And false assumptions used to inflate potential benefits of the canal were not being explained to Congress - openly, at least.

A sharp picture of corps management practices, dubious economic assumptions and political arm-wrestling that have allowed the Tennessee-Tombigbee to move ahead are emerging from preliminary action on the case.

The suit, brought in 1976 by a coalition of environmental groups, conservationists and the Louisville & Nashville Railroad, is expected to go to trial this summer in a U.S. District Court in Mississippi.

The Tenn-Tom waterway, as it is known familiarily in the South is planned to be a 260-mile-long barge canal in Mississippi, Alabama and Tennesse, shortening the inland water route to the Gulf of Mexico.

The canal has been a dream of economic development advocates in the region for nearly 200 years. And, since it was authorized by Congress in 1946 (although not funded). It has been a source of major controversy.

The history of the Tenn-Tom reads almost like a textbook on the traditional clash of economic interests, the environmental concerns and the community-rending disputes that dozens of corps projects have generated.

Railroads, which stand to lose business if the Tenn-Tom is built, have found it; environmentalists have challenged it repeatedly, including an unsuccessful suit in 1971; President Carter last year put it on his "review" list for possible termination, but backed off quickly when southern politicians raised a furor.

The Tenn-Tom has an enormous political constituency, abetted by the congressional seniority system that has elevated legislators from the Tenn-Tom region to control of the levers of public works building and spending.

Backing them up are the legislatures and governors of the five states that would derive the most from the canal.

A waterway authority was created by Alabama, Mississippi, Tennessee, Kentucky and Florida to beat the drums for the canal and keep it on track.

The idea behind the Tenn-Tom is to provide a new and shorter shipping route for barge companies moving bulk commodities - grain, coal, petroleum - between the Gulf of Mexico and inland river ports.

It would be achieved by tying the canal to the Tennessee River system on the north end and to the Tombigbee River system, ending at Mobile on the gulf, on the lower end.

Although the new route would parallel the Mississippi, it wouldcut hundreds of miles off the inland water connections between the gulf and such points as Minneapolis, Kansas City, Chicago, Cincinnati, Chattanooga, and Pittsburgh.

But the problem with the project always has been its cost, amplified by the need to build a chain of 17 locks and dams and to excavate a volume of dirt sufficient to fill six railroad trains stretching from New York to Los Angeles.

As the Corps of Engineers has calculated the potential benefits and cost over the years, the figures have only marginally justified construction of the canal.When it was authorized in 1946, the estimated cost was $116 million. Now the estimate is $1.8 billion.

The project, hung up on the narrow margin of benefits to cost, despite strong support in Congress, got no construction money until the Nixon administration, in what was seen as a political bow to the South, included the Tenn-Tom is the fiscal 1971 federal budget.

The steps used by the corps in coming up with the estimated benefits that justified the cost of construction of the Tenn-Tom are a central part of the Mississippi litigation.

Beyond that, however, the documents obtained by attorneys Jon T. (Rick) Brown of Washington and Joe Karaganis of Chicago raise a series of questions about the corps' activities.

Memos and letters from the corps indicate that Army engineers' attorneys and officials had serious doubts about the legality of their building a canal 300 feet wide, as opposed to the authorized width of at least 170 feet.

The doubts were expressed despite the fact that the chief of engineers in 1966 and Secretary of the Army Stanley Resor in 1967, citing "discretionary" power in the 1946 congressional authorization, said the Tenn-Tom project could proceed on the 300-foot basis.

Corps planners, in restudying the project between 1964 and 1966, apparently assumed the engineers could count on a 300-foot width all the way to Mobile - that is, including part of the existing Tombigbee navigation channel below the point where it would join the new canal.

The Tombigbee, however, is authorized by Congress to extend only to a 200-foot width. Thus, corps calculations projected benefits based on an illegal width. Figured on the 200-foot base, the project would not pay because it would not accommodate large-sized tows of barges; at 300 feet, it would pay.

Corps attorneys commented that the authority to build the canal 300 feet wide, rather than 170 feet, was "cloudy" and "doubtful" or possibly "nonexistent."

Another problem emerged in 1974, according to the documents, when the engineers discovered that the actual costs of the project were going to exceed the latest authorized cost: $732 million.

A basic cause appeared to be the large amount of earth excavation that was required - contract bids were coming in far higher than the corps anticipated.

By October 1974, the engineers knew the waterway would cost a minimum of $1.1 billion, some $400 million more than authorized. But in early 1975, the corps told Congress the new cost would be $815 million.

At that point, the secretary of the Army, then Howard H. (Bo) Callaway, became concerned about the overruns. He called the Army Audit Agency in to investigate.

The audit agency's report, not released to Congress, according to Brown and Karaganis, said the increased costs were not made public to reduce the "emotional impact" of any big cost increase and to negate the effect that the increase would have on the project's benefit-cost ratio."

The auditors warned that the anticipated state participation in sharing part of the cost might not materialize or might fall short. And, in any case, the auditors said the canal could pay its way only if the Tombigbee were widened.

Attorneys for the L&N Railroad, the Environmental Defense Fund and the other plaintiffs maintain that unnamed corps officials decided at a meeting in November 1975 to "proceed illegally" to build the 300-foot-wide channel for the canal, disregarding advice of corps lawyers and planning officials.

The Staats-Stennis exchange and the GAO involvement in the dispute over benefits and costs is another chapter in the story unfolding in the pretrial discovery actions in Mississippi.

Until this year, Stennis had been chairman of the Appropriations subcommittee for public works, which controls the pursestrings for corps projects. He also is chairman of the Armed Services Committee.

With most of the canal project scheduled to be built in eastern Mississippi, Stennis has been a principal congressional supporter of the canal.

According to the memos from the GAo files, Stennis and his staff expressed a keen interest in the agency's activities when they learned GAO was reviewing the procedures the corps used to calculate benefit-cost ratios.

One of GAO's findings was that the corps' methodology was "inaccurate" and "misleading." GAO's report was to be sent to the secretary of defense.

But the report never left GAO after Stennis, according to the memos, contacted Staats to ask if the agency actually planned to release its findings.

Efforts to reach Stennis and his chief staff aide, W. E. Cresswell, both of whom are mentioned in the GAO documents, were unsuccessful Thursday and Friday.

A spokesman for Staats said the comptroller general does not keep a log of telephone calls he receives, but that he did recall talking to Stennis about the report.

The spokesman explained that GAO has "a firm policy" of not conducting audits when litigation is involved in a case. On that basis, he said, the decision was made not to issue a report on the study.

Although the decision to hold up release of the report was made last fall, the litigation challenging the Tennessee-Tombigbee had been filed almost a year earlier.

Handwritten margin notes by one of the GAO memo writers say, "It seems this decision was made solely to keep from offending Stennis."