Retired Army general Paul Gorman told a Senate panel yesterday that when he was U.S. military commander for Latin America, based in Panama, he never received any credible information linking the Panamanian leader, Gen. Manuel Antonio Noriega, to drug trafficking.

Gorman, who headed the U.S. Southern Command in 1983-85, said U.S. Embassy officials in Panama repeatedly assured him Noriega was a "major contributor" to U.S. efforts to combat the drug trade.

But according to allegations in two U.S. federal indictments returned against Noriega last week, Noriega at the time had converted Panama into a safe haven for international drug traffickers smuggling narcotics into the United States.

Gorman said an investigation he initiated determined that Noriega was making a fortune from a wide array of "very, very shady" commercial ventures, including shipping, airlines and import-export businesses. Gorman said he had access to information from several U.S. military intelligence units based in Panama, but said his inquiry did not link Noriega to the drug trade.

After his Senate testimony, Gorman said that he had heard "rumors" but that "I had no solid reason to believe {Noriega} was engaged in any direct sense" in drug trafficking.

Gorman's testimony before the Senate Foreign Relations subcommittee on terrorism, narcotics and international communications is likely to fuel the controversy that has erupted over whether the U.S. government covered up or ignored allegations tying Noriega to drugs. Reagan administration officials have strongly denied these claims.

Congressional officials note that the reports of Noriega's cooperation received by Gorman are consistent with letters of praise written to Noriega by John C. Lawn, head of the U.S. Drug Enforcement Administration, and his two predecessors, Francis (Bud) Mullen Jr. and Peter B. Bensinger.

Administration officials in the last six months have been pressing Noriega, Panama's military commander and de facto ruler, to resign and permit restoration of democratic rule, a move that has prompted Noriega to charge that the indictments are part of a "systematic campaign" to destabilize Panama.

In a television interview broadcast Sunday on CBS's "60 Minutes" program, Noriega added a new element, charging that his troubles stemmed from his refusal to participate in an alleged secret U.S. plan to invade Nicaragua. Noriega said he learned of the plan in a December 1985 meeting with then-Vice Adm. John M. Poindexter, the president's national security adviser.

White House spokesman Marlin Fitzwater yesterday denied that the administration ever considered such a plan, calling Noriega's claims the "idle charges of a man indicted for drug-running."

Current and former U.S. officials have said Noriega, who headed Panama's military intelligence before becoming military chief in August 1983, had a close relationship with the Central Intelligence Agency and was viewed as an important link with Cuban leader Fidel Castro.

Gorman, who said he did not have a close relationship with Noriega, said "another agency of the government" sponsored trips by Noriega to the United States. One informed source said the unidentified agency cited by Gorman was the CIA.

Gorman said Noriega regularly reported to U.S. officials on his meetings with Castro, but added: "I don't think any of us paid much credence on what he said."

When he arrived in Panama in 1983, Gorman said, he unsuccessfully recommended that the Southern Command headquarters be removed from Panama because "we were, in effect, in the hands of this man {Noriega}."

"I would not do anything to irritate him, lest he indulge in one of his picayune acts of retribution," Gorman said.

The indictments allege that Noriega laundered large sums of money derived from U.S. drug sales between 1981 and 1986. Gorman said he did not learn of Noriega's alleged ties to drug-money laundering until after he retired from the military and worked as a consultant in 1986 to the President's Commission on Organized Crime.

Staff writer Bill McAllister contributed to this report.