The leadership of the National Rifle Association, long recognized as the toughest and least compromising lobby in town, is under attack from an internal faction charging that the organization is getting soft on gun control.

The revolution is being led by Neal Knox, who was ousted a year ago as head of the NRA's political and lobbying arm, the Institute for Legislative Action.

The power struggle will be resolved this weekend, when as many as 30,000 NRA members and supporters gather in Phoenix for the annual convention, which President Reagan is scheduled to address Friday night.

The turmoil has surfaced in charges and countercharges in publications read by gun owners and collectors around the country.

Knox has proposed a series of bylaw changes that could immediately oust the man who runs the NRA, Executive Vice President Harlon B. Carter, by cutting his term from five years to one. Another bylaw change would transform Knox's old job from an appointive to an elective position, presumably clearing the route for his return to power.

Knox also has circulated a series of vague charges about the current NRA leadership, calling Carter "dictatorial," hinting at fiscal mismanagement and--worst of all in the eyes of many members--suggesting that the NRA may be willing to compromise slightly on gun-control legislation.

Joe Tartaro, editor of the tabloid Gun Week, which has carried Knox's charges in paid inserts, says that untangling the intrigue and factionalism involved in the fight is almost impossible. "It's difficult even for a veteran NRA-watcher," he said.

In a telephone interview, Knox refused to elaborate on his charges. "It's nothing of any great interest to anyone outside the NRA," he said.

Knox's attacks have infuriated NRA officials, who point to such successes as the defeat of an anti-gun referendum in California and a doubling of the membership to 2.6 million since 1979.

Knox was at the center of the last major revolution, when suspected "soft-liners" and conservationists were turned out of office at the NRA's 1977 convention by a group known as the Federation of the NRA. Knox, a member of the federation, was largely responsible for putting Carter into the NRA leadership at that time.

Knox would not talk about his departure from NRA headquarters last year except to say it was not voluntary. His job was taken over by Warren Cassidy, 52, a former mayor of Lynn, Mass., who successfully led a 1976 drive to defeat a Massachusetts gun-control referendum.

Cassidy was outspoken about the firing, saying Knox was alienating supporters, especially on Capitol Hill.

"You don't swagger into an office and bluster and threaten retribution," Cassidy said. Rather than "branding" a congressman who doesn't vote with the NRA, he added, "we work with him, try to bring him along. If that doesn't work, eventually we would try to defeat him."

Knox also has been blamed for bringing the NRA into the battle over the 1979 nomination of Abner J. Mikva, a strong gun-control advocate, to the U.S. Court of Appeals. Cassidy said he believes that the NRA generally should not try to defeat judges. And in this case he says Knox "called in all the previous chips for a losing cause."

According to NRA officials, Knox's most grievous offense was his handling of the NRA campaign against the Bureau of Alcohol, Tobacco and Firearms (BATF), the enforcers of the federal gun laws. Critics say he managed simultaneously to embarrass the NRA, delight its enemies and enrage supporters at the White House and in Congress.

In the wake of the NRA campaign, which included a Knox-produced film describing the BATF as a "jackbooted group of fascists who are . . . a shame and a disgrace to our country," the Reagan administration decided to dismantle the bureau and distribute its functions to other Treasury divisions.

But when Knox discovered that gun-law enforcement was going to be turned over to the Secret Service, an agency far harder to criticize, he shifted into reverse to revive BATF. So far, the bureau lives.

Any thought that the NRA might be weakening on gun control is close to heresy for most members. Tartaro says there is a general feeling that the current administration may be more "conciliatory."

Knox claims in particular that Carter has been willing to compromise on a 14-day waiting period between buying a gun and picking it up that was approved by the Senate Judiciary Committee last year. The waiting period is intended to reduce crimes of passion and give law enforcement agencies time to check gun buyers' criminal and mental health records.

Cassidy says the NRA would never support a waiting period. He said the rumor apparently started because the NRA was using an old copy of the Senate committee bill in its lobbying efforts this year.