The small but colorful National Maritime Union is locked in a fierce internal power struggle featuring a band of dissidents and legal cross fire of charges that range from sexual misconduct to racketeering.

At the heart of the struggle is Kirby-Smith McDowell, a weathered former seaman built like a tug and with a piratic glint in his eye. After 30 years as a good soldier working his way up through union ranks, he has created a stir from Baltimore to Port Arthur, Tex., with his hard-fought challenge against NMU President Shannon Wall.

"The Wall must fall," growls McDowell, 56. "This is a grass-roots movement from within. What we're after is union democracy."

Takeover battles are rare inside American unions. Rebels snatched control of the coal miners' union from Tony Boyle in the 1970s; a steel union revolt failed a few years later, and a dissident group has existed inside the Teamsters union for several years.

McDowell never finished grammar school but says he has taken a Dale Carnegie course in getting along with people, as well as lessons in public speaking, collective bargaining and other subjects.

He and fellow dissidents charge that Wall and other union officials have mismanaged union funds, paid themselves excessive salaries and expenses, assessed members improperly to pay for it all and used threats and harassment to stay in control.

"We got 165 officers and staff spending $10 million a year," McDowell said. "It's going someplace . . . . Our membership is dwindling rapidly. Our benefits are going downhill. And the people in the ivory tower in New York are conducting business as if we were in a boom time."

Last year, Wall's salary was more than $136,000, not counting $9,400 he was reimbursed for expenses, according to copies of union documents filed with the Labor Department. McDowell points out that the figure is considerably more than the $110,000 annual salary of AFL-CIO President Lane Kirkland.

"I've said if I'm elected I'd cut my salary by 25 percent immediately," McDowell said, "and further, I'd have a referendum by the membership to set my salary and all the officers' salaries."

Robert Amon, a spokesman at NMU headquarters in New York, said Wall, who is attending a meeting in London, declined to comment on the dispute.

In the bygone "days of wine and roses," as one official put it, the union's members served as able-bodied seamen, cooks and stewards aboard the great U.S. oceangoing passenger ships. Peak membership is said to have been between 45,000 and 55,000.

Now the United States operates no passenger ships, and the union's members serve aboard the dwindling fleet of merchant vessels, operate the Panama Canal or work as security guards and cooks at military bases and other public facilities.

AFL-CIO records show that the union had 30,000 members in 1981. Wall's spokesman said the number was 50,000. "It's a disparity," he said by way of explanation.

Several thousand of those work in Panama for the Canal Commission. The U.S. Merchant Marine has only 18,870 jobs, of which one official said NMU members claim 5,500 of the most menial.

Wall, 64, is the NMU's second president in its 45-year history. Ten years ago, he succeeded union founder Joe Curran, a legend in the labor movement.

Last year, after McDowell mounted his challenge, Wall brought disciplinary action against him and suspended him from his 20-year job as the union's Houston port agent. McDowell and fellow dissident Albert Jackson of Wilmington, Del., retaliated with a multimillion-dollar lawsuit against Wall.

In the suit, the dissidents charge Wall and other officials with attempting to chill dissent and also with conspiring to operate and maintain control of the union "through a pattern of racketeering activity," including intimidation and threats. One allegation in court papers is that a Chicago NMU official once threatened to "blow McDowell away" with a pistol.

McDowell's cause was advanced March 1 when U.S. District Judge Charles S. Haight Jr. in New York directed that he be reinstated with back pay. The order confirmed earlier findings by a magistrate that Wall and union Secretary-Treasurer Thomas Martinez had violated the Landrum-Griffin Act and the union constitution "for the purpose of quashing McDowell's chances for electoral success against the incumbent leadership."

The Landrum-Griffin Act regulates internal union business, management of union funds and conduct of union elections.

Wall had acted against McDowell on the basis of charges that McDowell had given preferential job assignments to a woman union member with whom he was having an affair and to two of her relatives. After the woman ended the relationship, it was also charged by the woman and her relatives, McDowell discriminated against the three.

The magistrate cited "misconduct" by union officials in making the case against McDowell and concluded that the violations "did not arise from a good-faith procedural irregularity, but rather from a deliberate abrogation of fair procedures that was calculated to ensure that the membership would vote to accept the charges preferred against McDowell."

Spokesman Amon said last week that the court ruling did not address McDowell's guilt or innocence. McDowell denies the charges.

Since that ruling, the union leadership has twice extended dates for the next election of officers. Originally scheduled April 1, it is now set for Aug. 1 and, because of prolonged seagoing absences and members' far-flung whereabouts, the election period lasts two months.

The dissidents say the extension is in part an effort to give the membership time to forget about the court ruling.

Union officials announced that they postponed the election in order to "avoid the possibility of an election challenge on legal technicalities" that might necessitate a costly rerun. They blamed the delay on the need to conduct a referendum on amendments that would affect candidates' eligibility standards and supervision of union elections.

Meanwhile, union chapters in Philadelphia, Baltimore, Houston and Port Arthur, charging that the union leadership is corrupt, have voted in favor of placing the next union elections under supervision of an impartial arbitration association.

McDowell and Jackson have circulated a letter indicating that the union leadership planned to spend $300,000 on this year's elections. The dissidents argue that the American Arbitration Association can do the job for less than one-tenth that amount.

McDowell has been traveling from port to port, making his pitch to the members. He is personally financing his campaign and legal expenses, he said, helped by a cash settlement awarded him after his neck was broken in an accident.

He argues with some irony that he ought to know about corruption in the union since for decades he was on the inside. He won on slates headed by Wall in 1973 and 1978.

He became disenchanted in 1978, he says, when union leaders put pressure on him to "go out and hustle" more shipboard injury business for union lawyers. "Ambulance chasing," he said. "They were saying to me, 'You are not a member of the family.' "