President Reagan's "New Federalism" is in for some more tough sledding if the response of Republican mayors is any indication.

Cleveland Mayor George V. Voinovich, a businesslike Republican and frequent White House ally, said Reagan's effort to shift federal programs to the states is "bad public policy" that extends aid to those who least need it.

"I don't think it's doing what the ideologues in the White House think it should do," said Voinovich, the new president of the National League of Cities. "There's a growing sentiment that it's not working."

Cleveland, he said, received federal aid to exterminate rats until that program was folded into a health-care block grant to the states. "Now the state has a brochure on how to kill rats, and we don't get any money to kill them anymore," Voinovich said.

It is not surprising that most of the Democratic mayors who gathered here for the league's winter meeting are hostile to New Federalism. They were among the leading critics of Reagan's sweeping 1982 plan to transfer dozens of federal programs to states, calling it camouflage for further budget-cutting.

Despite their opposition, the administration succeeded in merging a number of programs into state block grants, and now White House officials are talking about extending this effort in Reagan's second term.

Voinovich, who looks and sounds like a midwestern banker, is the kind of Republican who might be expected to support the plan. He does not.

Voinovich's favorite example involves the $725 million summer youth-employment program.

Reagan folded the program into the Job Training Partnership Act and turned it over to the states, which distributed much of the money to small communities. Cleveland, Chicago and other big cities lost nearly half their summer-job funds and had to pressure Congress into voting another $100 million to make up the difference.

"The states gave the money to all these little towns that didn't need it, and the ones that really needed it didn't get it," Voinovich said. "Now there's a bigger constituency out there to keep that program, and everyone and his brother will be promoting it. The block-grant approach spreads the money around to too many places.

"It's stupid politics," he added. "It's the best way for the Reagan administration to keep Democratic governors in power. The governor gets the money and wraps a ribbon around it, and people think it's Ohio money."

Other mayors voiced more traditional criticisms. "In many areas, all we've done is to build a new bureaucracy at the state level," said Colorado Springs Mayor Robert M. Isaac (R).

St. Paul, Minn., Mayor George Latimer (D) said administration officials might use New Federalism "to dignify their effort to avoid national responsibilities, but it won't wash."

"Most local officials feel they don't get a fair shake from the states, and they'd rather deal with the feds," said Indianapolis Mayor William H. Hudnut III (R). "All this talk about sharing power and responsibility with Washington is fine, but if the money doesn't accompany it, you've got a huge problem."