Britain today ordered the deportation of 21 Libyan students described as threats to national security, and West Germany and Denmark said they would sharply reduce the size of Libyan diplomatic missions to their countries.

British officials said that the 21 Libyans, who were picked up in early-morning police raids throughout the country, were being detained until they could be deported, probably to Tripoli later this week.

Officials said that the Libyans were not accused of any specific crime, but had been under surveillance for some time as militants in Libyan student revolutionary groups. Although their expulsion was not directly related to events of the past week, Home Secretary Douglas Hurd said he had decided to act against them now "in light of the latest situation."

Belgium, the Netherlands and Luxembourg also expect to move next week to expel Libyan diplomats, Luxembourg Foreign Minister Jacques Poos said after meeting in Luxembourg today with his Benelux colleagues.

[Early Wednesday, Information Ministry officials said in Tripoli that all American and Western European journalists were being ordered out of Libya immediately, The Associated Press reported. The official said reporters would be taken to the airport within 10 minutes and that the move was in retaliation for the deportation of Libyans from Britain.]

The diplomatic reductions announced in West European capitals marked the first response to measures agreed to by the 12 European Community governments yesterday to inhibit Libya's ability to support terrorist activities abroad.

The EC nations agreed to reduce the size both of their embassies in Tripoli and of Libyan missions in Europe. The 12 also agreed to restrict the movements of remaining Libyan diplomats, to examine further restrictions on nondiplomat Libyans, and not to admit to their countries any Libyan expelled from another EC member nation.

Both Britain and the Reagan administration had hoped for tougher European moves against Libya, including the severing of all diplomatic relations with the Tripoli government. But Prime Minister Margaret Thatcher said today that the community "had gone a good deal further than previously."

British officials said privately they were "pretty satisfied with the outcome" of the community meeting, attributing what they saw as a change in attitude among a number of governments to a "focusing of attention" after last week's U.S. air attack against Libya.

Most of the moves agreed to at the meeting of community foreign ministers yesterday in Luxembourg had been tentatively decided in a gathering last Monday at The Hague, before the American raid. But that meeting was marked by dissension and the insistence of a number of members that any measures against Libya be coupled with an appeal for U.S. restraint.

At the same time, agreement at The Hague to restrict and reduce Libyan diplomatic missions had been considered discretionary by several of the community members. During yesterday's Luxembourg meeting, officials here said, governments including West Germany, France and Spain had declined to raise previously voiced objections to a tougher stand against Libya.

A number of those governments still are critical of the American attack. Some European diplomats have suggested that today's announcements reflect a desire to forestall further U.S. action rather than a wish to signal approval of last week's raid.

In Bonn, government officials said that West Germany intends to reduce the size of the Libyan mission there by more than half, and will cut back its own embassy staff in Tripoli to the bare minimum, Washington Post correspondent William Drozdiak reported.

The Libyan "People's Bureau" in Bonn employs 41 persons, including diplomats, and West German officials said they expected the staff to be cut "as soon as possible" by more than half, to about 15. West Germany's embassy in Tripoli is staffed by 22 persons, half of whom are accredited diplomats, and Foreign Ministry officials said personnel there will be cut to the absolute minimum required to look after the interests of approximately 1,400 West German residents in Libya.

Danish Foreign Minister Uffe Ellemann-Jensen said Denmark would expel several of its seven resident Libyan diplomats "as soon as possible," Reuter reported today from Copenhagen.

The Benelux countries have a total of seven diplomats in Libya, and the current Libyan contingent of 14 persons in the three countries is expected to be cut back at least to a level of parity.

At least one EC member, however, has indicated it is not prepared to take action based on yesterday's agreement. In a statement today in Athens, a government spokesman said that Greece still is awaiting "tangible proof . . . which has not emerged up to now" of Libyan complicity in terrorism.

The position of France remained unclear. A Foreign Ministry statement today said it was "too early" for Paris to say whether it will reduce Libyan missions there, Reuter reported. Officials reportedly stressed that France already had taken action against Libyan-inspired violence by expelling two diplomats and four temporary Libyan residents on national security grounds.

In a statement to the House of Commons here today, Thatcher said she is considering further action against some of the estimated 7,000 Libyans currently living in Britain.

Among them are about 250 trainee pilots and aircraft engineering students who Thatcher said would come under special scrutiny. One of the students detained today for deportation, Adil Masood, 23, was studying at an Oxfordshire pilots' school. Early this month, Libyan radio broadcast a telephone conversation from Britain to Tripoli in which Masood pledged that he and other student pilots were prepared to undertake suicide missions against U.S. targets.

[In Brussels, the European Community announced that the sale to Libya of cut-price butter will not go ahead. EC officials in Washington said the elimination of the subsidy was a political action and is probably the first time such a measure has been directed against Libya.]