The Supreme Court, moving away from one of the traditional prohibitions of U.S. patent law, ruled for the first time yesterday that an invention based on a computer program may be patented.

The 5-to-4 decisions is expected to break a logjam of about 3,000 patent applicatins for inventions involving computer programs, which are mathematical formulas. The government has always fought such patents on the grounds that they grant monopoly rights over something that should be freely shared: laws of physics and mathematics that exist in nature and are discovered, not created, by man.

The justices agreed yesterday that a formula or computer program, by itself, could not be patented. But when a new use for it creates a new process, the court said, it may be patentable.

Large corporations with research and development teams sought such a ruling as an incentive to innovation. The government, which said there was plenty of innovation in the computer field, feared such a ruling.

Antitrust lawyers claim that it will enable skillful patent application draftsmen to win patent protection for computer programs just be defining them as part of a larger process. They fear that corporations might be able to patent and nomopolize the sightest modifications in software technology through this means.

It was the second major Supreme Court ruling in the past year to open up the patent laws to new technology. Last term, the justices ruled patentable the products of genetic engineering. That ruling, also fought by government antitrust lawyers, was accommpanied by delcarations about "incentives" for technological "progress."

No such language appeared with yesterday's opinion, written by Justice William Rehnquist, who noted that the court had ruled computer programs unpatentable in a 1978 decision. The application at issue yesterday was not for a new formula, but for an old one, put to new use.

The Federal-Mogul company of Detroit makes synthetic rubber seals designed to prevent oil leakage in automobile and railroad engines and wheel bearings. The synthetic rubber must be heated, or cured, in a mold from its initial mushy form to a hardness that will effectively seal. If it isn't cured sufficiently, it will leak.

For years, the curing process involved the kind of guesswork familiar to chefs with erratic ovens. Technicians-knew that rubber had to be cured at a certian temperature for a certain amount of time. But they were never certin of the exact temperature inside the heating mold. To be certain the seals were properly cured, Federal-Mogul overcured, cooked them a few extra seconds or minutes, even though this hurt productivity.

Two corporate scientists, James R. Iehr II and Theordore A. Lutton, solved the problem with a thermostatic device that uses a specially programmed computer to take constant temperature measurements and limits the cure time to only that required. The program was based on a 50-year-old formula called, after its Swiss discoverer, the Arrhenius equation.

Federal-Mogul can always use the technique in its own factories. But the corporation wanted to sell the idea to others for a profit. If a company does this without a patent, competitors can simply copy the technique. With a patent, competitors must pay Federal-Mogul for the use.

A patent application must pass at least two tests: it must claim something new and something patentable. If it is not considered patentable subject matter, the application will not even be examined to see if it is new.

Congress, which recently said computer programs can be copyrighted, has never decided specifically whether they can be patented.

The patent office rejected Federal-Mogul's application as unpatentable subject matter, determining that the only thing that made the process new was the use of the equation. A patent on the process would therefore patent the equation. The Court of Customs and Patent Appeals reversed the patent office, and the Justice Department appealed to the Supreme Court.

Rehnquist said that the company was not seeking to patent an equation bu itself, but a process employing the equation. A process does not become unpatentable "simply because it uses mathematical formula, computer program or digital computer. . . ."

The company, he said, seeks "only to foreclose from others the use of that equation in conjunction with all the other steps in their claimed process."

The Federal-Mogul invention does not automatically receive a patent under yesterday's ruling. Patent officials must still consider whether the process is, in fact, new.

The court did not rule on a second related computer program case involving a Honeywell, Inc., patent application argued at the same time as yesterday's case. A later opinion on that could confuse yesterday's ruling since Chief Justice Warren Burger, in the majority in yesterday's 5-to-4 decision, did not participate in consideration of the Honeywell case.

Justice John Paul Stevens wrote the dissent in the Federal-Mogul case and was joined by William Brennan. Thurgood Marshall and Harry Blackmun. aThey said that no "program-related invention" should be patentable if it is enterly dependent on the utilization of the computer.