Two influential governors challenged the Reagan administration and Congress today to move beyond "vague, woolly rhetoric" and arrange a bold swap of education and welfare responsibilities between the federal government and the states.

Gov. Lamar Alexander (R) of Tennessee and Gov. Bruce Babbitt (D) of Arizona suggested that the federal government give up its education subsidies and let the states carry that burden themselves, while relieving the states of their share of the Medicaid program for the needy and the aged.

Richard Williamson, President Reagan's assistant for intergovernmental relations, and Robert Carleson, a White House policy adviser, pointed out that Reagan had long opposed federal aid to education but was equally committed to making welfare a state responsibility.

"We don't want a confrontational situation," Williamson said. "The president has his views and they have theirs."

The proposal to swap about $22 billion of spending between state and federal budgets is not a new one.

But its bipartisan introduction at the start of the annual meeting of the National Governors Association in this seaside gambling resort dramatized the impatience many governors feel with what Babbitt called "the total inconsistency" of Washington's efforts to share power with the states.

That impatience poses a potential political problem for President Reagan, who had the help of the governors in overcoming resistance from Democratic mayors and congressmen in his effort this year to cut federal grants to states and cities while moving from narrow categorical to broad block grants.

Congress gave Reagan everything he wanted in budget cuts, but less than he sought in added flexibility for the states.

That led Georgia Gov. George Busbee (D), outgoing chairman of the association, to predict today that states next year will face "the cutback in the level of their services" or what he called "a massive increase in taxes."

Gov. Richard A. Snelling (R) of Vermont, next year's chairman of the association, suggested that states have a "high risk of failing this year" in adjusting to the new block grant system.

Snelling told a discussion session on handling block grants that it would be a disaster if the grants were judged on the basis of one year alone because next year there will be "a very, very narrow window for flexibility" to accommodate cuts running up to 25 percent.

He warned the governors that it would take longer for the block grant system to be fully effective.

It was a theme taken up by Gov. Edmund G. (Jerry) Brown Jr. (D) of California, who urged colleagues to take time to study the three-year impact of the block grant changes and try to ensure that the positive aspects of decentralization were not "stigmatized" by cuts in existing state programs.

Several governors said they believed the system would provide opportunities for new approaches to economic and social problems.

Alexander said states would now consider how to encourage greater volunteer effort. Volunteers, he said, could free funds to help those who cannot care for themselves.

Gov. James B. Hunt Jr. (D) of North Carolina spoke of encouraging more volunteer effort.

Other Democratic governors were unequivocal about the effect of cuts. Gov. John V. Evans (D) of Idaho said his state was in "dire straits." Gov. Hugh Carey (D) of New York said the block grant system had not really been changed, it had just been cut. The loss in such programs as drug abuse control cannot be made up, he said.

But Busbee struck a conciliatory tone, saying Reagan "has changed the direction of a federal aid system that was becoming increasingly rigid and fragmented. We have been, and we will continue to be, his partners in these efforts."

He added, however, that governors "feel strongly that federalism must be a two-way street," and called for more effective "joint determination" by Congress, the administration and the states of the next steps in the process.

Williamson, speaking for the administration, responded in kind, saying, "We will go back to Capitol Hill on block grants and seek the flexibility we did not get this year.

"The president feels a great deal of satisfaction at the progress we've made."

The Alexander-Babbitt initiative was an effort to start such a discussion on two of the biggest and most controversial programs where the federal government and the states are now both involved. Busbee gave the proposal his blessing as "a step in the right direction."

In broad terms, the two governors proposed that the federal government phase out about $10 billion of aid to education and pick up about $12 billion the states now spend as their share of the federal Medicaid program.

Babbitt, who decried the "vague, woolly rhetoric" on federalism, said federal grants pay only 8 percent of the national school bill, but that "relatively small contribution has resulted in federal dominance, to the detriment of education."

The proposal has been endorsed for many years by the Advisory Commission on Intergovernmental Relations, on which both Alexander and Babbitt serve.

But it faces two formidable problems in Washington. Reagan has long advocated that welfare is a state responsibility, not one for the national government. And education interest groups would make a major fight in Congress against any move to abolish federal aid to education.

But Alexander said the "sorting out" is needed "to make it clear to the taxpayer who is to blame if a program doesn't work."

The governor's executive committee approved its first standing committee on state-local relations in an effort to ease the controversy that developed this year between mayors and governors over the distribution of funds in the new block grants.

In a prelude to this evening's formal opening of the meeting, the Democratic governors held a $500-a-couple swimming poolside brunch that reportedly raised $100,000 for the national and New Jersey parties.

Former vice president Walter Mondale told the reception that the "ravaging of the economy by high interest rates" would help a Democratic comeback but warned that Democrats would need a "coherent, understandable fair plan for America's future" to regain power.