The leaders of 22 rich and poor nations meeting here agreed tonight to support "a round of global negotiations" in the United Nations on economic problems. But they failed to reach agreement on how or when to proceed.

The meeting of the heads of state and other top officials of eight rich Northern countries and 14 poor Southern nations thus achieved one basic goal that had been set at the start by the South -- a continuation of the dialogue on economic issues between the two groups within the United Nations.

The United States, which has long opposed using a U.N. forum as a framework for "global negotiations," was a hold-out against establishing a specific formula for the precise form and content.

In a final press conference, cochairman Pierre Elliott Trudeau, prime minister of Canada, said, "We failed to get agreement on what the next step would be." But he insisted that the Cancun outcome represented forward "movement" because the United States had joined in the call for global negotiations in the United Nations.

Trudeau said the decision to leave the next step to a vaguely defined "mutually agreed" procedure "begs the question." The Canadian prime minister was among those who wanted to send the issue to the United Nations for immediate debate. He indicated that it was the U.S. refusal to accept this plan that thwarted it.

The other chairman of the meeting, Mexican President Jose Lopez Portillo, stressed the positive. He noted that the talks had been "open" and "constructive" and had produced among the 22 leaders a "genuine spirit of confidence" in each other.

"The new task we face is to build on that confidence, to translate words into progress," he said.

Nonetheless, it was clear that deep ideological divisions remain between the United States and virtually all of the 21 other nations represented here on how quickly and how completely global economic problems should be handed to the United Nations for discussion.

Since the question of global negotiations was first raised in 1979, the United States has been unwilling to debate specific economic issues in the United Nations General Assembly, where it can be outvoted by the Third World nations. But President Reagan surprised much of the gathering yesterday by promising that if a series of four restrictive principles were accepted, "the United States will be willing to engage in a new preparatory process to see what may be achieved."

The major American concession today was agreement that global negotiations could be held at the United Nations. "What we haven't agreed to is to go to the United Nations and be bound by a majority vote," a senior U.S. official added.

The statement that the leaders agreed on tonight said, "The heads of government confirm the desirability of supporting at the United Nations a consensus to launch global negotiations on a basis to be mutually agreed on and in circumstances offering the prospect of meaningful progress. Some countries insisted that the competence of the special agency should not be affected."

A senior Reagan administration official said he expects preparatory talks at the United Nations on global negotiations will begin as early as December or January. He added that the United States had played an informal role in drafting the language read by Trudeau and Lopez Portillo at tonight's joint press conference. Their statement was not a formal communique, since the United States had insisted that no communiques be issued following the summit.

Although many of the poor nations believe no practical alternative exists to throwing what is a global economic problem to the main global legislature -- the United Nations -- they eagerly seized today on the small opening offered by Reagan yesterday. Trudeau also praised the Reagan initiative, and Canada and Mexico scheduled an afternoon session today devoted exclusively to a discussion of what global negotiations mean to the different countries.

"A door has been opened," one well-informed Mexican said. "There are differences between those who want radical change, and those who want to push the door wider open and walk through it."

Nigerian Foreign Minister Ishaya Shaiibu Audu said, "All one can say at the moment is that President Reagan has not shut the door." Audu added that Reagan had set "very tough conditions," which he hoped would be softened in further negotiations.

"I believe there is a universal commitment to succeed no matter how small the success will be," Audu told reporters. He said that the road to global negotiations will be tough and tortuous and "anybody who expects quick, easy solutions will be disappointed."

Brazilian Foreign Minister Ramiro Saraiva Guerreiro described Reagan's speech as "a little evolution." He was asked if Cancun had become a contest with the United States pitted against the other 21 nations. "It's never clear," he replied, "but certainly they are one extreme."

A Yugoslav put it this way, "The fact is that the U.S. is accepting global negotiating if not global negotiations. What is the alternative for the South? It would be to have 22 new resolutions in the U.N. and see nothing happening."

In his suite at the Sheraton Hotel this morning, Reagan was asked if he supports global negotiations.

"If there are those there, and possibly there are, who by global negotiations interpret that to mean some gigantic new international bureaucracy to be in charge, we would be opposed to that," he replied. "If global negotiations mean that we continue negotiations as to how all of us can help resolve these problems, we're perfectly willing to."

Reagan and his chief aides, including Secretary of State Alexander M. Haig Jr. and Treasury Secretary Donald T. Regan, came here to stress trade not aid, economic growth not a redistribution of wealth, and the private market not new major government-to-government involvement. And essentially they have made no concessions in these basic concepts of Reaganomics.

Philippine Prime Minister Cesar Virata said in a telephone interview that since the industrialized nations' summit last summer in Ottawa when the United States stonewalled the Canadian effort to get global negotiations on track -- even to the point of forcing the final communique to use a lower case "g" and "n," Reagan, Haig and Regan have made a series of clarifying statements, culminating in the president's approach yesterday and his acceptance of the summit leaders' statement today.

As many of the nations of the South here see it, an important part of the movement forward has been generated by the leadership of the Common Market, in the form of statements by Great Britain, France and West Germany. These countries said at Cancun that no significant new global economic growth can be made without the involvement of the Third World. In a change from the Ottawa summit, the three European nations have publicly moved toward strong support of a global economic dialogue.

On the other hand, American officials say somewhat tartly that despite the public posture, the major European nations privately are not unhappy to have the United States out front, developing conditions that may make it practical to have a meaningful global dialogue.

"Most of these countries can, and are happy to, put the burden on us," said one U.S. official. In fact, on one key issue that the United States and its wealthy allies stress -- the need to keep open trading markets -- the Western Europeans are clearly far behind the United States in accepting exports from the poor countries, notably and importantly in textiles.

The United States has made clear repeatedly in the private sessions with the other 21 delegations and in briefings with reporters that in attempting to solve global economic problems, it prefers to stick to specific issues.

At one point yesterday, Haig showed his displeasure for the basic themes of the Brandt report -- a plan widely supported by the delegations from the South -- by saying that the commission headed by the former West German chancellor proposed to "transfer resources in a wholesale way -- if you will, a transfer of existing shortages from the 'haves' to the 'have nots.' That's a conception we reject. What we are seeking is creating the conditions for global economic growth, and we feel that one of the most important vehicles for that is trade."

Despite the rather murky language of the four "essential understandings" that Reagan set out yesterday as prerequisites for American participation in some kind of global dialogue, it has become quite clear exactly what the United States will not agree to under any circumstances.

First, the United States rejects the South's demand, spelled out in the Brandt Commission report, for a "new international economic order." Instead, it proposes additional help from the rich nations that will improve the present system. The precise nature and form of this help has not been spelled out at Cancun.

Second, the United States wants the South to give up a long-sought after "reform" of the World Bank and International Monetary Fund, which the United States has argued would deprive the rich countries, who are the major donors of funds, from their control of the power structure. Treasury Secretary Regan has been fond of saying during the past few days that the United States cannot be expected to approve a switch of international financial activity to a one-nation, one-vote system in the United Nations, which would give the continent of Africa 56 votes as against only three for North America.

Third, the United States flatly rejects the pressure from its rich allies as well as recipient poor countries to boost its subsidized aid to as much as 1 percent, or even 0.7 percent of gross national product, at a time of budget stringency at home. The Reagan administration considers that this is unrealistic, even at a time that other nations, including Japan, are pledging increases in their subsidized aid commitments.

Fourth, the Reagan administration opposes increasing aid to those countries that refuse to take a major role in improving their own conditions, whether that means controlling excessive inflation or controlling excessive birth rates.

West Germany, which made friends in the South by its strong support at Cancun for global negotiations and also for a new energy-lending affiliate of the World Bank, backed the United States on this point. West German Foreign Minister Hans-Dietrich Genscher noted that "none of the forms of aid . . . can replace efforts made by the developing countries themselves . . . . "