The Treasury Department announced yesterday it has begun a formal investigation into charges that Japan may be "dumping" carbon steel products in the United States - a move that ultimately could lead to imposition of compensatory duties.

The move came in response to a petition filed last month by the U.S. Steel Corp., the nation's largest steel producer, alleging that six major Japanese steel producers were selling their products here below the prices they charge at home - in violation of trade law.

The action was the first since the Carter administration began encouraging U.S. steelmakers to take advantage of existing anti-dumping laws as an alternative to seeking new import quotas in order to deal with foreign competition.

In a second announcement, the department said it also would initiate an investigation into allegations that French steelmakers were dumping carbon steel wire rod here. The complaint was filed by the Georgetown Steel Corp. and the Georgetown Texas Steel Corp.

The moves came as, separately, a top AFL-CIO official warned that organized labor is seriously considering pushing for enactment of mandatory import quotas unless trade conditions "improve" over the next several months.

Lane Kirkland, secretary-treasury of the federation, said labor leaders will try to revive a measure similar to the Burke-Hartke bill of the early 1970s, which would have mandated the imposition of stiff import quotas unless the administration was able to negotiate voluntary restraints.

The Burke-Hartke bill was opposed both by the Nixon administration and by the Democratic congressional leadership. Economists and high officials warned it could lead to a new international trade war.

The Treasury's announcement, expectedly, was hailed by the U.S. steel industry. In Pittsburgh, a spokesman for the U.S. Steel Corp, urged the government to "expedite its investigation and quickly report its conclusions."

Without giving any details on the case, the spokesman said the company also plans to file a complaint soon against the British Steel Corp. The Treasury has been besieged by petitions from steelmakers since the administration suggested the dumping route a few weeks ago.

The Treasury's action yesterday does not guarantee that the Japanese firms will be found guilty of dumping here. The Treasury investigation will take a minimum of six months. The department then must make a tentative determination. Three months after that, it issues a final report.

The case filed by U.S. Steel involves four separate products - sheet, plate, pipe and tubing, and structurals.

The French case, centering on sales of carbon steel wire rod, involves only $38 million in imports. However, the petition is the first initiated against a member of the European Common Market since the new drive began.