President Reagan is considering asking Japan to voluntarily cut its sales of machine tools in the United States, threatening to set quotas otherwise under national-security provisions of U.S. trade laws that haven't been used before, administration officials said yesterday.

The import limits could last for as long as five years to preserve the ability of the import-battered domestic tool industry to produce machines needed to make weapons, the sources said.

"The decision is going to be made by President Reagan within the next couple of weeks," said one administration official, who has been pushing for action on a three-year-old petition by American machine-tool makers to win trade protection on national-security grounds.

Although several countries supply machine tools to the American market, any import restraints are likely to fall on Japan, which is the largest supplier of general-use machine tools to the United States.

European machine tools, which come largely from West Germany and Switzerland, are such specialized products that they do not compete directly with U.S. products. Japanese newspapers are speculating that the Ministry of International Trade and Industry is preparing for the restraints by seeking estimates from major machine-tool makers in that country of their export projections for this year.

U.S. machine-tool makers and their congressional allies have argued that foreign manufacturers produce about three-fourths of the state-of-the-art, computer-controlled lathes and machinery centers, necessary for manufacturing weapons ranging from missiles to rifles.

"The very foundation of our national security and economic well-being depend upon the skill and capacity of the machine-tool industry," 27 House Republicans, including Minority Leader Robert H. Michel (Ill.), said in a January letter to President Reagan. There is a "serious threat to the national security posed by our growing dependence on imports for high-technology defense-sensitive machinery," they added.

Commerce Secretary Malcolm Baldrige, who two years ago this week recommended that sharp limits be placed on imports of machine tools on national-security grounds, is pressing for the voluntary restraints on Japanese imports.

His original proposal, which never went to the full cabinet, was far stronger. It called for banning 90 percent of all imports, and effectively would have eliminated Japanese products from the United States.

That recommendation ran into sharp opposition from free-trade advocates within the administration, who sought to bury the issue by keeping it from a presidential decision.

"You had stalling because people thought it was a bad issue that could be stalled away," said an official who favors the import curbs.

The machine-tool industry's petition was brought forcefully to the attention of White house Chief of Staff Donald T. Reagan in December, when he was seeking support for tax overhaul from House Republicans.

They quickly reminded him of the buried recommendation to help U.S. machine-tool makers, and he promised to resurrect the issue.

Since then, it has moved toward the front burner of administration trade issues.

Secretary of State George P. Shultz met last week with three Republican House members -- Nancy Johnson (Conn.), Henry Hyde (Ill.) and Lynn Martin (Ill.) -- on the issue, and U.S. Trade Representative Clayton Yeutter publicly criticized the National Security Council for delaying a resolution of the industry petition.

Under questioning by Rep. Barbara B. Kennelly (D-Conn.) at a House Ways and Means Committee hearing last month, Yeutter said, "True to form, the National Security Council has not yet given me a date for a meeting on machine tools , and I cannot defend that at all.

"But I can be a little more optimistic than that because I really think we will do this within the next two to three weeks. I think we are finally nearing the conclusion of this process, which has been indefensively procrastinated."

He was scheduled to meet last week with the president's national security adviser, Rear Adm. John M. Poindexter, but the meeting was postponed because of developments in the Philippines.

The idea of so-called voluntary restraints on the part of the Japanese is considered more palatable to the free-trade ideology of the Reagan administration than Baldrige's original recommendation of sharp limits on imports through quotas and tariffs.

Rep. Johnson said Defense Secretary Caspar Weinberger told her he opposed import limits, but appeared more sympathetic to voluntary restraints, even if they are agreed to under the threat of imposed quotas.

Nose-counters within the administration and on Capitol Hill are unsure what the NSC will recommend to the president. "It's still up in the air," said one administration official.

In the three years since the National Machine Tool Builders Association filed its petition for import restraints, sales of foreign machine tools have increased steadily, going from 26.4 percent of the U.S. market in 1982 to about 43 percent last year.

The value of Japanese imports tripled in the same period, jumping from $535 million in 1982 to about $1.5 billion last year.