The British government's offer of limited home rule was overwhelmingly rejected by Wales and only narrowly approved by voters in Scotland in Thursday's referendum elections in the two ancient Celtic kingdoms.

The referendum results, whih were tabulated and announced today, kill the plan for an elected assembly in Wales and leave the final decision on a similar assembly for Scotland in the hands of the British Parliament because the Scottish majority in favor of the assembly was small.

The vote shows that the Scottish and Welsh nationalist movements, which seek eventual independence for the two regions, are still unable to mobilize broadly based support for even limited self-government. They ae left, however, with sufficient concentrations of loyalists to remain healthy minority parties with elected members in Parliament and the potential to force some decentralization of government in Britain.

The outcome was a serious setback for British Prime Minister James Callaghan, whose minority Labor Party government had pushed the home rule proposals through Parliament and had campaigned hard for them in Scotland and Wales. The fate of the Scottish assembly proposal in Parliament in the next few weeks also may determine how much longer Callaghan's embattled government remains in power in Britain.

In Wales, the proposal for an assembly with authority over education, housing, health and social welfare was rejected by a resounding 4-to-1 margin. Labor Party voters, the dominant force in Welsh politics, and opposition Conservatives combined in the 80 percent of the voters who said "no." Only Welsh-speaking advocates fo separatism voted "yes" in large numbers.

In Scotland, the proposal for a more powerful assembly there won by less than 80,000 votes out of nearly 2.4 million cast. The "yes" vote amounted to only 5.16 percent of those who turned out and 33 percent of the entire Scottish electorate.

That is far below the 40 percent of the total electorate required to implement the act creating the Scottish assembly automatically. An order to repeal the legislation now must be put before Parliament for a vote that will decide whether Scotland will get the assembly anyway.

Callaghan now must decide whether he will commit his government to trying to win the House of Commons vote to create a Scottish assembly. He would be opposed by the Conservatives and by some Labor Party dissidents who strongly disapproved of the assembly proposal and lacked the 40 percent requirement onto the legislaion.

If Callaghan loses the vote, his Labor government also could well lose a subsequent vote of confidence, which would force immediate national elections at a time when Labor trails the Conservatives badly in public opinion polls.

If Callaghan decided to duck such a confrontation by agreeing that the Scottish assembly proposal should die because the "yes" vote was too far short of the target 40 percent, he would immediatley lost the support in Commons of 13 Scottish Naionalist members of Parliament. Without their votes and those of the three Welsh Nationalist members of Parliament, Callaghan would still be in danger of losing a vote of confidence.

Callaghan would not comment today, but government sources indicated that he may take his time making his decision to keep the Scottish Nationalists on the string. The first formal government discussion of the referendum results is slated for next Thursday's regular Cabinet meeting.

The Labor government's secretary of state for Scotland, Bruce Millan, said today, "I'm disappointed. There was a majority vote for 'yes,' but it was not as decisive as I would like it to be."

Conservative Party leader Margaret Thatcher, who is now expected to push hard to topple Callaghan's government in Parliament, called the referendum result "a clear and decisive verdict."

"When only one in three Scots favor the government's proposal for an assembly," she said, "surely this is no basis for implementing the Scotland Act. A major constitutional change requires overwhelming support."

This was echoed by Tam Dalyell, a leader of the dissidnet Labor members of Parliament who oppose the Scottish assembly.

"Are we going to embark on a constitutional change that many of us believe will break up the United Kingdom," he asked, "when only one-third of Scotland bothered to vote 'yes'?"

The leader of the separatist Scottish National Party in Parliament, Donald Stewart, made clear, however, that he thought the small majority in favor of the assembly was sufficient and that his party expected Callaghan to push for its implementation within the next two weeks in Parliament.

"I wish the margin had been higher," Stewart said, but "the issue is decided by the people who bother to turn out. We have a majority and therefore and government shoudl deliver the assembly."

"We won," insisted Margo MacDonald, vice president of the Scottish National Party. "Seventy-seven thousand more Scots voted for the assembly than voted against it. We expect the Labor government to do everything it can to get approval for the assembly in Parliament. If it can't, then we want a general election."

A poll for Britain's commercial television network showed that voters indentifying themselves as Scottish Nationalists voted overwhelmingly for the assembly, while most Conservatives voted against it and the Labor voters were split.

In Wales, the 80 percent vote against an assembly matches the pro-portion of Emgligh speakers in the population, many of whom resent the seccuess that Welsh-speaking nationalists have had in creating a bilingual society there. This apparent backlash outweighed an expensive Labor Party effort to persuade its Engligh-speaking faithful to vote "yes."

Gwynfor Evans, leader of the Welsh nationalist party, Plaid Cymru, was angry today about Labor's failure.

"Although a small number of Labor leadres worked hard and sincerely," he said, "this must be a bitter day for those who thought they could serve the interest of Wales through the Labor Party."

Evans did not say whether the three Plaid Cymru members of Parliament now would join the Conservaties in moving against Callaghan in the House of Commons. However, referring to what he called a collapse of morale among Labor Party workers in Wales, he added, "It is likely that this collapse will be repeated in an early general election."

The Labor government's secretary of state for Wales, John Morris, admitted that the Welsh result was a "massive rejection of government policy. When you see an elephant on your doorstep, you recognize it and concede defeat. It is not only a loss, it is a very serious loss."

The opposition Conservative spokesman on Wales, Nicholas Edwards, declared that "any governmemt that has been defeated on so important an issue must consider its position as a result."

This is the theme the Conservatives are expected to repeat with vigor in Parliament, beginning next week. Their confidence has been swelled both by the referendum results and by impressive victories Conservative candidates won in two elections for empty seats in Parliament yesterday.

The Conservatives increased their share of the vote in each of the two constituencies by more than 10 percent. That reflects pubilc opinion polls showing the Conservatives as 19 percent ahead of Labor nationally, following a winter of labor strife and controversy over the home rule plans for Scotland and Wales.