The heralded new House rule limiting members' outside earned income took effect last year, but some members have found escape hatches that make the rule meaningless, Congressional Quarterly says in its annual review of financial disclosure reports filed on Capitol Hill.

The rule, which defines earned income as salaries, fees, commissions, and honoraria for making speeches of writing articles, set a limit of 15 percent of congressional pay, or $8,743 in 1979, when a legislator's salary was $58,290.

On its face, the rule seemed to confront several members with a sharp loss of income.

A case in point is Rep. Robert H. Mollohan (D-W.Va). In 1978 he drew a $72,901 salary from Interstate Fabricators and Constructions Inc. of which he was secretarty, director, and half-owner. In 1979 his salary shrank to $8,704 or $39 under the limit.

But Mollohan adjusted painlessly to his seeming $64,197 loss of income when a second firm, Coal Constructors, Inc., which he also owns, increased its divident payout to him from $7,000 in 1978 to between $50,001 and $100,000 in 1979. Under the House rule, dividends, along with spouses' pay, don't count, being defined as "unearned" income.

The spouses' pay exemption was invoked by Rep. James H. Quillen (R-Tenn.). In 1978, two of the five insurance companies he founded paid him a salary of $48,570, while in 1979 they paid him $41,370 less. CQ reported: a"His loss was mitigated, however, by his wife's growing stature in the Quillen companies. Where she had previously received salaries from two of The Quillen companies. Where she had previously received salaries from two of the companies, in 1979 she began receiving salaries from all five firms."

Just how much the quintet of companies paid her isn't known, because House rules do not require reporting of a spouse's income, and Quillen's press aide declined to say.

None of this really came as a surprise. During debate on the 15 percent rule, critics warned that one of its key goals, eliminating conflicts of interest, would not be achieved for wealthy members who get unearned income.

As Rep. Bill Frenzel (R-Minn.) put it at the time, "if you own the company, you may be all right; but if you just work for it, you probably become unethical."

Even more bluntly, Rep. Thomas L. Ashley (D-Ohio) denounced the rule as "the stupidest thing that ever came down the pike," protesting that it "quarantines certain kinds of income while allowing other kinds."

But the rule had a second major purpose, which was to reduce the total time House members spend moonlighting. And, say Rep. Richardson Preyer (D-N.C.) who headed a committee that drafted the rules, the rule has succeeded in doing just that. At the same time, he acknowledges, "Many members have rearranged the form in which they got [their] return."

The CQ survey of financial disclosure reports, which is made annually, is an exhaustive, 77-page compendium focusing on the new restrictions on outside earned income.

In learning to live with the restrictions, CQ concluded in its Aug. 23 issue, "members of Congress are like dieters struggling to shed some extra pounds. They seem to be cutting down on rich foods and desserts, but their weight still keeps going up."

Here are some highlights of the CQ report.

Outside Earned Income. In 1978, 204 representataives reported $5.8 million; in 1979, with the 15 percent lid on, only 165 members listed any, and the total received, $2.7 million, was a drop of more than 50 percent.

Unearned Income. It must be disclosed, though only in broad categories, not precise sums, and it appeared to rise allmost as steeply as earned income fell. In 1978, 363 House members reported unearned income of between $6.5 million and $14.2 million; in 1979, 385 members reported between $8 million and $17 million.

The Senate, by contrast, showed steady gains in outside earned income. In 1978, 28 senators reported $632,810; in 1979, 40 reported $1.6 million. Unlike the House, however the Senate imposes no curb on earned income.

Honoraria. In 1978, the first year in which House members were required to disclose these payments, which are for services not legally or traditionally required to be compensated, 249 members listed $1.2 million. In 1979, despite the earned-income limit, honoraria grew, with 287 members listing $1.5 million. t

A representative may retain up to $1,000 of any single honorarium. If he is paid more he must donate the excess to charity or return it to the donor. Last year, 37 representatives donated or refunded $104,691. That lowered the total for honoraria to $1.3 million -- still about $100,000 over the 1978 level.

For senators, the 1976 election law sets a $2,000 maximum for a single honorarium and an annual limit of $25,000 for honoraria. In 1978, 75 senators received $1.1 million; in 1979, 85 got $1.2 million.

Quirks exist in all of this. "For example," CQ said, "members may not accept gifts worth more than $100 from lobbyists with a direct interest in legislation. But if the member makes a speech to the group or renders other 'personal services,' he may accept reimbursements for food, lodging, transportation and other 'necessary expenses' associated with the speechmaking."

Some specifics:

Particularly in the Senate, many wealthy members have huge unearned incomes. Last year, for exampple, Sen. Claiborne Pell (D-R.I.) reported unearned income of $586,334 to $977,817. In the House, Rep. James . Collins (R.-Tex) realized between $304,189 and $841,000 in unearned income.

Leading a list of the 15 organizations that gave the most in honoraria in 1979 was the Grocery Manufacturers of America, which paid out $26,520 to 22 legislators. Next came the Chamber of Commerce of the United States ($26,290, 45 legislators) and the Brookings Institution ($20,750, 74 legislators). Then came 10 trade associations and one company, Pfizer In. ($17,250, 23 legislators). Coming in last was the AFL-CIO ($11,500, 11 legislators).

In 1979, when the Senate and House Banking committees considered controversial legislation on how financial institutions do business, banking and credit-union groups paid honoraria of $37,700 to seven members of the Senate unit. The senior Republican, Sen. Jake Garn (Utah), led the list getting $13,800. In the House, committee Chairman Henry S. Reuss (D.-Wis.) got $5,744 of the total of $47,145 given to 27 members.

Some legislators are "repeat performers," becoming what CQ termed "annual fixtures at groups' meeting or conventions." Since 1973, CQ found, 61 senators and 122 House members have spoken before one or more interest groups at least twice. Sen. John Tower (R-Tex.) spoke to the American Bankers Association in 1975, 1976, 1977, and 1979, getting a $1,000 honorarium each time.

Sen. Henry M. Jackson (D-Wash.) was the Capitol Hill honoraria champion last year, earning $30,950. He donated the entire sum to charity. In he House, the leader was Rep. Dan Rostenkowski (D-Ill). He earned $29,494, but, under the 15 percent rule, was able to keep only $8,743. He gave the balance to charity.

At least 44 representatives and four senators had some 1979 income from law practices. Rep. Frank J. Guarini (D-N.J.) listed $1.5 million in "accounts receivable" for legal work he said he did before taking his seat last year. Some House memmbers, while no longer actively practicing, allow their old firms to use their names, in possible violation of the American Bar Association's advisory code of conduct.

In Joliet, Ill., a law-firm partnership still uses the name of Rep. George M. O'Brien (R.-Ill.) although he sold his share in it. "I don't give a damn what the ABA says," O'Brien told CQ.