Lukens Steel Co. filed the first formal dumping complaint yesterday against foreign steel producers since the Treasury put its special system of minimum prices for steel imports to protect domestic manufacturers into effect last May.

The Treasury Department announced yesterday that Lukens charged five European countries with selling plate steel below "fair value" in the United States.

Lukens, a middle-sized manufacturer based in Coatesville, Pa., cited the United Kingdom, Germany, Italy, France and Belgium. The company complaint said that all five countries were selling plate steel in the U.S. at prices less than the so-called guidance prices established by the European Community for steel sales within Europe.

The Lukens anti-dumping complaint came only a day after the president of the nation's largest steel makers charged that as much as a quarter of the steel imported into the U.S. in September and October came in at prices below the minimum, or trigger, prices established by the Treasury.

David M. Roderick. of U.S. Steel Corp. said that unless the government does a better job enforcing the trigger price mechanism and unless steel imports decline soon, U.S. Steel will have no "choice but to file formal dumping charges."

U.S. Steel, and several other large steel companies, withdrew their dumping complaints from the Treasury after the administration instituted the special trigger price system.

Under the trigger price system, the Treasury itself institutes a speeded-up dumping investigation if steel comes into the country below trigger price. The agency has begun several of these investigations.

The trigger prices are based on the price of making steel in Japan, reputedly the world's most efficient producer, and shipping it to the U.S.

Treasury officials have said that they cannot run the trigger price system and deal with major dumping complaints at the same time.

Both U.S. Steel and Bethlehem Steel Corp., the second biggest producer, are reported to have formal dumping complaints that they could file with the Treasury at any time. Major producers have been reluctant to file the suits, however, because they do not want to antagonize the Carter administration nor destroy the trigger price system.

But steel companies are getting irritated because steel imports do not seem to be declining, despite the trigger price mechanism.

Through November, steel imports totalled 19.8 million tons, compared with 17.2 million tons for the first 11 months of 1977.

At the same time, however, steel demand has been very high and major steel companies are recording big profits and high levels of capacity utilization.

One steel source noted that the Lukens complaint may not lead the way for other dumping suits, because Lukens makes only steel plates, a product category that has been hard hit by imports. Lukens does not have other products to take up the slack in plate.

Last year, imports accounted for 38.5 percent of total domestic plate purchases, while during the first nine months of this year imports captured 43.7 percent of the market.

Lukens in its complaint does not charge that foreign steel companies are selling below cost, as most earlier dumping suits in the steel industry did, but merely that they are selling below home-market price.

Lukens said that the sales below home-market price ranged from 6 percent in the case of the West Germans to 20 percent for Italy.

A normal antidumping investigation, such as the type filed by Lukens, can take 18 months to two years to complete. Under the trigger price system, the speeded-up investigations are supposed to take no more than nine months.