A group of prominent economists yesterday denounced the proposed balanced budget amendment as an unworkable, politically inspired scheme whose passage would increase economic instability, not reduce it.

Meanwhile, the Senate rejected the first attempt to change the wording of the constitutional amendment by a margin that, although not conclusive, indicated that proponents may have trouble when the measure comes to final passage.

The economists, seeking to counter the surge of support for the amendment, argued that its adoption could lead to budgetary chaos and force Congress into unwise actions in efforts to circumvent the amendment's effects.

At a news conference, Gardner Ackley, chairman of the Council of Economic Advisers in the Johnson administration, presented a list of 172 names of economists opposing the amendment.Six Nobel prize winners were on the list. Ackley said he had heard of only "a handful" of economists who have supported the amendment.

"No one can speak for a profession," Ackley said. "None of us can speak for the professional economists of the United States, but I think this comes pretty close. Professional economists . . . regard the proposed balanced budget amendment overwhelmingly as bad public policy and are opposed to it."

Five of the six economists at the news conference were Democrats and liberals. The exception, Hendrik Houthhaker of Harvard University, a CEA member under President Nixon, said he would favor some "suitably worded limitation on expenditures.

"However, the constitutional amendment is the wrong way to go," Houthhaker said. " It is as phony as a $3 bill. It's phony because if Congress wants to balance the budget it can vote a balanced budget. If the president wants to balance the budget he can propose a balanced budget."

James Tobin of Yale University, the most recent winner of the Nobel prize in economics, called the amendment a serious hazard to the nation's economic health. He suggested that the members of Congress who support it be asked how they would in fact balance the budget.

Tobin said the amendment would increase economic instability and invite what he called "cosmetic accounting." Moreover, he said, the theory behind the amendment, which lays the blame for high inflation and slow economic growth on the size of government, is "unsubstantiated and dubious."

Economic consultant Robert Nathan called the amendment "not only cynical but an abdication of responsibility."

Asked why there were not more Republican economists opposing the amendment, Houthhaker replied, "I don't think it is a partisan issue at all. Most economists, I am sorry to say, are Democrats."

The amendment's Senate supporters have established a strategy of opposing all but a selected few floor amendments in order to make final congressional approval easier.

In the first floor test yesterday, they defeated, by a vote of 53 to 45, one offered by Sen. Wendell H. Ford (D-Ky.) that would have required the president to submit a balanced budget to Congress each year.

The narrow margin indicated that proponents may find it difficult to reach the 67 votes needed for final passage without making concessions to senators currently on the fence.

"I think it shows that they have real problems and not just minor problems," Ford told reporters later. "My hard-headed count is that they could not pass it today. There are some very thoughtful senators who don't play politics who are beginning to come down against the constitutional amendment."

Ford is officially undecided but indicated he could be induced to back the constitutional measure if another of his amendments is accepted.

President Reagan, who has enthusiastically endorsed the proposal, reportedly asked several sponsors yesterday to accept a change in the language.

As presently drafted, it would permit an exception to balanced budgets in case the country is at war.

According to senators present at the White House meeting, Reagan asked that the exception be broadened to include national emergencies short of actual war.

A number of opponents are preparing to introduce several amendments in the next few days and it appeared likely the final vote will come next week.