President Carter's plan for persuading states to help their distressed cities is fine as far as it goes, but it does not provide enough money to be a real incentive, two governors told Congress yesterday.

Govs. Michael S. Dukakis of Massachusetts and Julian Carroll of Kentucky also said the program should be broadened to reach all states rather than "only a handful" and should be run by a White House office, not by the Department of Housing and Urban Development.

Dukakis told the Senate subcommittee on intergovernmental relations that while the Carter bill would encourage state fiscal and institutional reform, it "falls short of its promise to make state governments vital and active members of the new partnership" to save urban America.

The Carter bill provides $400 million ovr two years for states that come up with strategies to give more fiscal aid to hard-pressed cities. HUD, which would administer the grants, has informally estimated that only 15 to 20 states would be eligible in the first year.

Dukakis suggested that the funding be doubled to $800 million, while Carroll advocated $1 billion.

Dukakis told reporters after the hearing that the incentive grants and other urban aid should be coordinated by a "small action office" created by the White House.

He praised HUD Secretary Patricia Roberts Harris as "first rate" but argued that "any single Cabinet officer would have a difficult time trying to pull everything together."

Sen. Edmund S. Muskie, subcommittee chairman, agreed that a single office should be set up to coordinate the entire great system, reduce the regulatory burden, and simplify grant application procedures.

"Today we have a creaking structure of aid programs that overlap, that conflict, that are snarled in red tape and regulations, that do not keep up with changing needs, that are poorly administered - in short, a sturcture badly in need of renovation," the Maine Democrat said.

And aide said later Muskie was speaking of long-term reform and was not addressing the question of how to administer the state incentives.

Harris testified that "the administration believes that this program can best be administered by HUD's because of its experience in dealing with urban problems. "It's important that we do not create new entities to do what present entities can do," she said.