The 1981 Nobel Peace Prize was awarded today, for the second time, to the Office of the United Nations High Commissioner for Refugees for its aid to the "enormous and growing number of refugees" who form part of a "veritable flood of human catastrophe and suffering" across the globe.

The Norwegian Nobel Committee, announcing the award in Oslo, took particular note of the fact that the Geneva-based office of the high commissioner has carried out its growing workload "despite the many political difficulties" strewn in its path. The U.N. refugee agency cares for about 10 million of the approximately 14 million refugees in the world today.

Poul Hartling, the U.N. refugee commissioner and a former Danish prime minister, said in Geneva that today's award "is a statement to the world's refugees that you are not forgotten."

The award this year is worth a record $180,000. The first time the U.N. agency won the coveted prize was in 1954, for its work in resettling displaced Europeans after World War II.

In its citation today, the Nobel Committee stressed the work of the high commissioner's office in aiding refugees from countries that were only emerging from colonial domination when the original award was received. "In recent years," it said, "we have . . . watched the mass exodus of people fleeing by land and sea from Vietnam" and "today we have, in addition, 2 million refugees from Afghanistan and an equal number from Ethiopia." Nearly half of the refugees being cared for by the U.N. agency this year come from African nations, and they are mainly in camps in Somalia and Sudan.

The committee concluded that "the stream of refugees creates serious problems in the relations between states, and for this reason the activities of the Office of the High Commissioner for Refugees serve the interests of humanity and peace as well."

Hartling, 67, said, "I take the award as confirmation of fundamental humanitarian principles and the right of all of us, especially refugees, to live in peace and dignity." The prize, he said, "comes to us at a time of crisis in the refugee world: today . . . the refugee problem knows no geographical or political bounds."

When the U.N. agency won its first peace prize in 1954, it had about 150 employes and a budget of around $5 million. Today, it employs 1,600 persons in Geneva and in offices in more than 80 countries and works with a budget of $460 million.

The agency has helped resettle more than a million "boat people" from Vietnam, Laos and Cambodia since 1975 and last week it reported the resettlement of 600,000 Zimbabweans. In earlier years it aided people displaced by conflict in Cyprus and refugees from Ugandan dictator Idi Amin.

Hartling, who took up his post in 1978, admitted to reporters today that "the Nobel Prize couldn't have come at a better time for our organization," alluding to the meeting here this week of its executive committee, at which a number of countries, including the United States, have criticized the agency's operations. Hartling said he hoped the good news from Oslo would encourage all of those "in a position to help refugees . . . to redouble their efforts."

In a speech to the executive committee yesterday, U.S. Ambassador to the United Nations in Geneva Gerald Helman said that the United States could only continue to "do our part" in supporting the organization's activities "despite very genuine budgetary restraints" if the relief agency recognized the "elements of more effective operations."

These elements, Helman said, included better management, sharper distinctions between aid to refugees and development assistance -- which is not related to refugee relief, according to the U.S. government -- and improved cooperation with other U.N. bodies and with voluntary relief agencies.

In 1981, the United States contributed $130 million to the agency's budget. For 1982, when the the agency says it will require more funding to carry out its mission, Helman said, "I cannot predict how much the United States will pledge." He added, "I am confident, however, that we will do our part."

The U.N. relief agency won over 17 other organizations and 77 individual candidates nominated. Individual nominees included Lech Walesa, leader of the Polish trade union federation Solidarity, the British foreign secretary, Lord Carrington, Prime Minister Robert Mugabe of Zimbabwe and former president Jimmy Carter.

The award ceremonies will take place in Stockholm Dec. 10, the anniversary of the death of Alfred Nobel, the inventor or dynamite, who set up the prizes in his will, and the the 80th anniversary of the first award.