North and South Yemen, whose remote border war led the United States to a display of firmness with emergency arms shipments to the pro-Western North, announced yesterday that the 3-week-old conflict is over.

An accord, reached in the Nroth Yemeni capital of Sanaa and announced by the official Iraqi News Agency, followed a truce worked out several days ago by Arab League mediators. It stipulates that forces of both sides will begin withdrawing today and pull back fully behind their own borders by next Friday, the Iraqi agency said.

If respected, the agreement marks a formal end to this round of the fighting even as the United States mounts an accelerated delivery program for a $400 million weapons package to North Yemen, including F5E jet fighters, tanks and antiaircraft guns accompanied by a limited number of American military advisers.

The hostilities have been sputtering for five years, however, and yesterday's accord may turn out to be only another pause. The recent fighting erupted with a South Yemeni invasion Feb. 23. It had been dying away over the last several days as the Arab League team of Syrian, Iraqi and Jordanian officials moved between Sanaa and Aden, the capital of the Marxist South.

Announcement of the withdrawal pledge coincided with another sharp attack from Rep. Les Aspin (D-Wis.) criticizing the Carter administration's decision to make the conflict an occasion for demonstrating to Saudi Arabia and its Persian Gulf allies that Washington is resolved to protect friends in the region.

"... the situation in Yemen is becoming less and less of an emergency every day, and North Yemen is hardly about to be overwhelmed by a fullscale South Yemeni invasion in any event," Aspin said in a statement. "The rush simply seems highly rash and possibly dangerous. Given the Arab League's recent steps toward peace and the Nroth Yemeni Army's inability to use many of these weapons in short time."

Aspin, who voiced similar complaints in a House speech earlier this week, referred to President Carter's use of provisions of arms control legislation that allow him to waive congressional notice if he declares the arms exports are necessary to meet an emergency.

U.S. officials said the emergency shipments were responding to repeated appeals from Saudi Arabia, which is closely allied to Nroth Yemen and eager to prevent any increase in the influence of the Marxist rulers of South Yemen. The Carter administration has been seeking to display particular resolve in its support of the Saudis since the fall of the Iranian shah, which was widely interpreted as a setback for U.S. policy and an indication that Washington may not be reliable in a crisis.

In addition, sources said, there was a feeling within the administration that it was time to signal to the Soviet Union that the United States would not tolerate gains by the Sovietbacked South Yemenis that could be interpreted as lack of resolve by the United States.

"I think an awful lot of this was directed at the Soviet Union," said one U.S. official.

South Yemen, the former British protectorate of Aden at the southern tip of the Arabian Peninsula, has moved closer to Moscow since Abdul Fatah Ismail, a doctrinaire Marxist, came to power in a bloody coup last June. U.S. analysts estimate 800 to 1,000 Soviet military advisers are attached to his army, 500 to 700 Cubans to his militia and about 100 East Germans to the internal security forces.

North Yemen, which has sent a million of its citizens to work in the oilfueled Saudi Arabian economy, is strongly influenced by the conservative Saudis and heavily dependent on their largesse fro development aid. Saudi Arabia is financing the U.S. arms shipments.

North and South Yemen both have pledged repeatedly to work toward unity. At the same time, ideological hostility, tribal feuds and border conflicts have prompted frequent skirmishes along the mountainous, ill-defined border that twists up from the Bab el-Mandab Strait toward the desert fastness of Saudi Arabia's Empty Quarter.

The strategic location of the Yemens -- beside Bab el-Mandab, or Gate of Lamentations, at the exit from the Suez Canal and the Red Sea into the Indian Ocean -- gives added importance to the Saudi concerns over North Yemen, U.S. analysts said.

Reports in the Arab press say the Saudi concern also is heightened by doubts about North Yemeni leader, Ali Abdullah Saleh, who is said to lack popular appeal among the various tribes and even within his own government. He took over after Maj. Ahmed Hussein Ghashmi was killed last June by a bomb concealed in a briefcase brought into his office by a special envoy from South Yemen.

Within four months, a mutinous military unit shelled Saleh's headquarters in a coup attempt that failed.

Observers had been expecting the South Yemeni government to try to capitalize on these weaknesses since Ismail took control and gradually strengthened his hold on the military and government.

Ismail seized power from President Salim Robaya Ali and had him executed after the June coup in what analysts said was a reaction to Robaya Ali's moves to respond to encouragement from Saudi Arabia to settle differences with North Yemen.