Two conservative political organizations are refusing to pay nearly $4 million in disputed fees to the direct-mail firm run by New Right fund-raiser Richard A. Viguerie, including one group that had 750,000 of its letters wind up in a Chicago dump last year.

Officials of Ruff-PAC, a Washington-based committee that authorized the mailing, said they lost hundreds of thousands of dollars in potential contributions and are withholding about $531,000 until the dispute is settled with Viguerie, a contender for the Republican nomination for lieutenant governor in Virginia.

A second group, the National Conservative Political Action Committee, said it is withholding $3.2 million of $3.3 million in fees charged by Viguerie, whose firm mailed 12 million fund-raising letters for the group last year. NCPAC contends the letters cost far more than they raised.

The disputed fees are an indication that Viguerie, who has gained a national reputation for direct-mail appeals that have poured millions of dollars into conservative political causes, is not always the "wizard" of fund raising that many have called him.

Larry Sabato, a Virginia political analyst and author of a book on political action committees, said that while Viguerie originated the direct-mail concept, he is no longer the most competitive in the fast-growing business. And some critics of direct-mail operations are raising questions about the effectiveness of the expensive technique.

In an interview yesterday at his Falls Church office, Viguerie said the problem accounts are a small part of his business, which handled 102 million letters for about 35 clients last year.

"That's part of business," Viguerie said. "Look at the big corporations . . . look at GM. They are constantly calling cars back."

Viguerie declined to provide further details of his business operations, but said he would disclose more information on his clients and finances later in the lieutenant governor's campaign.

Viguerie, who surprised Virginia GOP leaders by jumping into the race, is competing with four candidates in his first bid for state office. He unsuccessfully ran for vice president in 1976 as an independent.

U.S. postal inspectors in Chicago, called in by both Viguerie and Ruff-PAC officials after the committee got a poor response to its mailing, are investigating the letters found in the dump. Viguerie said he did not know how the letters went astray.

Federal Election Commission records, which provide one of the few glimpses into Viguerie's network of business enterprises, show that at least seven conservative political action committees have yet to pay Viguerie more than $4 million for a variety of direct-mail services. Of that, $3.7 million is being disputed by the clients.

Ruff-PAC is contesting its bill because 750,000 of its 2.48 million 1984 campaign brochures promoting President Reagan were found shredded in the Chicago dump.

"It appears as though 2.2 million of them were never delivered," said Neil Blair, president of Ruff -- PAC, who said he based his calculations on the meager contributions from the mailing. "I don't have the foggiest notion what happened."

Viguerie said yesterday that he subcontracted the work to UARCO, a large bulk mail processor that in turn contracted with Lee Enterprises of Chicago to mail the brochures.

James Brown, chief counsel for UARCO, said in a telephone interview this week that his firm did not know what happened. He said it was involved in the disputed mailing but declined further comment because of the postal investigation.

David Sajovic, a vice president of Lee Enterprises, said in a telephone interview that he's confident his employes did nothing wrong but declined further comment.

Chicago postal inspector James Kelley said his agents are investigating "the falsifying of mail forms" which appeared to be signed by three postal workers as well as the subcontractors.

NCPAC, which raised more than $14 million in 1984, said in its latest FEC report that it is $4.1 million in debt, with three-quarters of that owed to Viguerie. The NCPAC report noted that the Viguerie amounts were "subject to contest and verification."

Both NCPAC and Viguerie declined to comment on the specifics of the dispute, which involved a mailing of more than 10 million letters in support of Reagan. Viguerie said that "we wish we hadn't mailed so many letters" and that the response was less than expected.

Overall, Viguerie said his company is doing well and discounted suggestions that he is financially strapped. "The money is coming in, probably slower that we would like," said Viguerie.

Susan Petersen, a Houston political consultant and former executive director of Ruff-PAC, said there are problems with the direct-mail concept used by Viguerie and others. "We have direct-mail companies raising money for themselves," she said. "It's a sham, it's inefficient and a poor use of money."

Viguerie objected to what he said was too much emphasis on whether the huge volume of direct mail raises large sums of money. He said fund raising is a small part of the overall messages that promote conservative causes and have helped build the conservative movement.

Howard Phillips, director of the Conservative Caucus, said Viguerie's firm has been crucial to his organization's success and that Viguerie has absorbed more than $2 million in losses on direct-mail campaigns for the group.

"We've very happy with our relationship with Richard Viguerie," Phillips said.

"I sometimes do let my zeal to help solve problems" override business decisions, Viguerie said. "The purpose of the Viguerie Co. is to advance the causes I believe in."