A powerful business lobby with close connections in the Carter administration made a strong case yesterday that the business community might be worse off, rather than better as popularly supposed, if the so-called "double taxation of dividends" were ended.

This warning came from the Business Roundtable, a relatively new lobby of corporate leaders. The chairman is Irving R. Shapiro, head of duPont Co. and a strong supporter of President Carter.

The Roundtable said other forms of tax relief, especially a 6-point reduction in the present 48 per cent corporate tax rate, would be "more direct" and "preferred" by the business community.

Co-chairmen of the Roundatable are John D. deButts of American Telephone & Telegraph and Reginald H. Jones of General Electric, who also is working with the administration to formulate an anti-inflation policy acceptable to business.

The Roundtable's views on double taxation of dividends came in a series of position papers on tax revision sent to members of Congress and the administration. They were developed by a taxation committee headed by Jones.

The administration is publicly committed to consider ways of eliminating double taxation of dividends. But there have been private hints lately that the administration feels it may have a bear by the tail because - as the Roundtable report indicates - there are seemingly as many disadvantages as advantages in each potential change.

One of the key problems is that the economic effects of double taxation on business investments, savings and inflation have never been determined.

The double tax 'bite' arises because a corporation's income is taxed first at the corporate level, and again when its dividends to shareholders are taxed at the individual level. One possible revision would allow corporations a deduction from their taxes for dividends paid or a credit to stock-holders for the tax the corporation paid.

But the Roundtable articulated what has become the view of many business leaders. It said "there is a real risk that the business community would pay a prohibitive price for removal of this inequity."

Basically, the risk forseen by the Roundtable is that the huge tax loss to the government resulting from the end of double taxation might be offset by a reduction in other tax benefits now given to business, especially accelerated depreciation and the tax credit for investment.

What the Roundtable would like is more of such relieg as improvement in the investment credit and lower basic tax rates.

"There is no question that elimination of double taxation of dividends is a desirable objective," the report said. "If it could be done in a way which would enchance the availability of investment capital in the corporate sector by improving corporate profitability and return from investments in equity securities, the business community would strongly support such a program.

"On the other hand, other forms of tax reduction . . . together with a permanent investment tax credit, or a reduction in the corporate income tax rate, should have a more direct and measurable effect on corporate profitability, and, if so, would be preferred if they are achievable."

The Roundtable made other tax suggestions, but did not advocate accepting them all because the revenue loss would be too great.

All were keyed to the theme that the united States must increase the share of resources devoted to capital investment. To reach Carter's economic goals, the Roundtable said, business investment for the rest of the decade must be accelerated to 13 per cent of gross national product from last year's 9.3 per cent.

But real return on investment (adjusted for inflation) dwindled from 9.9 per cent in 1965 to 3.7 per cent in 1976. "hardly an incentive to invest at today's cost of money," the Roundtable papers said.