The Carter administration has decided to speed the movement of much larger number of Vietnamese and Cambodian refugees to the United States from crowded camps in Southeast Asia, according to administration and congressional sources.

After several days of urgent discussions. Justice and State department officials are considering plans that could effectively double for one year the present annual quota of 25,000 Southeast Asian refugees allowed into the United States. They also are acting to speed the processing of refugees who qualify to come here.

In the first formal move, Attorney General Griffin Bell yesterday began consultations with congressional leaders on an immediate emergency expansion of the quota to allow into the United States 2,500 more "boat peope" who have fled Vietnam by water.

This action was intended to help persuade the government of Malaysia to give temporary sanctuary to more than 2,500 refugees from Vietnam - mostly ethnic Chinese - who are stranded on the freighter Hai Hong two miles off Port Klang, Malaysia.

Until the ambassadors from the United States, France and Canada assured Malaysian officials in Kuala Lumpur yesterday that their countries would try to provide homes for the refugees crowded aboard the Hai Hong, the Malaysian government was planning to have the disabled freighter towed back out into international waters.

In continuing negotiations with representatives of the three governments and of the U.N. high commissioner for refugees last night, Malaysian officials insisted that the refugees must be directly removed from the Hai Hong to the countries where they are to be resettled.

U.S. officials, however, want them taken first to camps in Malaysia where about 35,000 other "boat people" are already awaiting resettlement. The Hai Fong refugees would be processed for possible emigration to the United States after those who are already in the camps, according to State Department officials.

Malaysia does not want to add the 2,500 aboard the Hai Hong to the already overcrowded shantytown refugee camps on its eastern coast.

Its stated reason is that the Hai Hong's passengers, believed to be former merchants and their families, are not genuine refugees. Instead, the Malaysian government has said, they paid perhaps $5 million in gold and U.S. dollars to buy their way out of Vietnam, to pay Hong Kong entrepreneurs to outfit the Hai Hong, and pick them up off the Vietnamese coast.

The frieghter, which first tried unsuccessfully to unload its human cargo in Indonesia, has been in Malaysian waters for 10 days and may have been on the sea for nearly 2 month.Two babies reportedly were born on the ship, where the refugees are packed tightly together with little food, water or medicine in increasingly unsanitary conditions.

The plight of the people aboard the Hai Hong became a symbol this week of the rapidly worsening refugee problem in Southeast Asia.The State Department official in charge of refugee affairs, Henry Cushing, said yesterday that the number of refugees leaving Vietnam by sea alone has increased from 2,000 in March to 6,000 in July to an estimated 20,000 this month. At least another 130,000 refugees remain in camps in Thailand, Indonesia and elsewhere.

Faced with these numbers and worldwide attention to the fate of the Hai Hong, high-level Justice and State department officials began exploring a number of options for bringing more refugees into the United States.

In addition to the immediate expansion of the refugee quotas by 2,500 to help ease the Hai Hong emergency, they are considering an addition of 7,500 per year to the 25,000 annual quota for two years for refugees from Cambodia and a still further expansion of this year's quota by perhaps 15,000 to help reduce the backlog of "boat people" in Southeast Asian refugee camps.

Final decisions by States and Justice officials and the approval of congressional leaders are still required. Congress has specifically authorized the attorney general to set the quota for the number of reguees who can be given special "parole" entry into the United States. But Attorney General Bell has been expected to consult with the House and Senate Judiciary committees before making changes in the quota.

Some officials in Washington expressed concern yesterday that opening the door here to more refugees will only encourage the Vietnamese government to push out more who are willing to pay. These officials said they were particularly worried about more large boatloads of refugees such as those aboard the Hai Hong.

The U.N. high commissioner for refugees, Paul Hartling, has called a meeting in Geneva Dec 11 and 12 of "all interested government" - including the United States and Vietnam - to discuss the "boat people" and other Vietnamese refugee problems. It is not yet known whether Vietnam will send a representative.

The United States has taken in more than a quarter of million Southeast Asian refugees since the Vietnam war ended. Officials of France, which also had been heavily involved in Indochina, say they have taken in more than 45,000 refugees from Southeast Asia since 1975.

Senator Edward M. Kennedy (D-Mass.) has been urging Carter administration officials to act quickly to allow more refugees into the United States. Kennedy, who is expected to become chairman of the Senate Judiciary Committee when the new Congress convenes next year, will introduce legislation to overhaul U.S. immigration laws, including provisions for the entry of refugees.

To qualify for entry now, a Southeast Asian refugee mush have relatives here, have had ties to the U.S. government during the war or have some other American connection.

The drama of the Hai Hong reportedly began when entrepreneurs in Hong Kong or Singapore, knowing that refugees in Vietnam would pay to get out, bought the empty, 30-year-old, 1,500 ton freighter in Singapore, and sent it to pick up the refugees off the Vietnam coast and deposit them wherever possible.

As negotiations continued in Kuala Lumpur last night, the fate of the refugees on the frieghter was still in doubt. They were, however, reported to be receiving food and water and other supplies from the Malaysian government while talks went on.