A key part of President Carter's urban program is in deep trouble and may be defeated in the House next week, Capitol Hill sources said yesterday.

But a White House official, conceding that "that vote would go against us today," said, "We're pulling out all the stops. We're doing everything in our power to turn it around."

The legislation in peril is the supplementary fiscal aid bill, which would give communities with sluggish growth and high unemployment a total of $1 billion a year for two years.

It suffered a serious blow Tuesday when Democratic members of the House intergovernmental relations subcommittee voted 5 to 3 in caucus against it.

The vote came as a surprise to some administration and urban lobbyists, who were predicting only a few weeks ago that of the 15-part urban program the fiscal aid proposal had the best chance of passage.

"If they'd take the gloves off their toes, they could count to 20," said one House aide, expressin disgust at apparent White House inability to foresee the bill's lack of support in Congress.

The crucial vote will come Wednesday when the full 13-member subcommittee takes up the legislation.

White House lobbyists and senior Treasury Department officials have spent the last few days trying to persuade key members of the sumcommittee to support the bill. The administration strategists will meet Monday to review their efforts and decide whether Carter or Vice President Mondale should call subcommittee members personally.

Meanwhile, the National League of Cities and the U.S. Conference of Mayors have sent out calls for help.

"It's our heaviest lobbying effort in years," said Tom Tatum, deputy director of federal relations for the league.

"We've phoned or telegraphed mayors in about 70 of the largest cities, asking them to contact the subcommittee and others in Congress."

The conference on Thursday alerted mayors of the nation's 800 cities over 30,000 in population, urging them to contact Democratic and Republican members of the subcommittee and full Government Operations Committee.

"This bill is do-or-die for many key cities," said Dorothy Brodie, an assistant executive director of the conference. "They are counting on it to replace thr anti-recession aid that they've been getting, and some of them have put it in their fiscal 1979 budgets." New York would get $140.4 million; Philadelphia, about $24 million; Los Angeles, nearly $16 million; Chicago, $17 million and Washington, about $10 million, she said.

Under the anti-recession program that expires Sept. 30, states and localities have been getting $1.3 billion this year based on their jobless rates. The new program would eliminate aid to states and send it to localities if their unemployment is over 4.5 percent or if they meet two of the following three "growth lag" criteria: if their population, par capita income, or employment is growing more slowly than that of the rest of the country.

At least one subcommittee member who opposes the bill called it "an irresponsible program" because the criteria do not reflect the true fiscal condition of communities.

Some members also critized the fact that the new measure would aid 26,000 localities, at least 7,000 more than are getting anti-recession funds now.

"It's clearly not targeted to communities in distress," said one Hill sources. "What the administration has cooked up is something for the silly season. They should have kept it for Halloween."

Another aide, Alvin From, staff director of the intergovernmental relations subcommittee, said subcommittee chairman Edmund S. Muskie (D-Maine) also is concerned about the lack of targeting in the Carter proposal and favors a measure, costin under $400 million, that would direct aid to local governments with jobless rates of more than 6 percent.