While Congress and the Securities and Exchange Commission watch from the sidelines, courts in Delaware and New York are rewriting the rules governing the high-stakes merger game.

Amid the growth of questionable business practices that has accompanied the surge in corporate takeovers, the courts are going further than ever before in second-guessing decisions by corporate directors, according to several merger experts.

In the process, judges are dictating the outcome of multibillion-dollar takeover battles. Within hours of three recent court rulings, fights for control of Unocal Corp., Revlon Inc. and SCM Corp. were decided. And Texaco Inc. was hit by a record judgment in excess of $10.5 billion when a Texas court found that it improperly interfered with Pennzoil Co.'s planned merger with Getty Oil Co.

One of the most controversial elements in recent rulings is the new challenge judges are posing to business decisions by corporate directors. Not long ago, under an important doctrine known as the "business judgment rule," prudent decisions by corporate directors acting in good faith were generally free from judicial interference.

The legal theory behind the business-judgment rule is that corporate directors are in a better position than judges to make informed business decisions. Although this doctrine still applies, the business-judgment rule has been altered by the Second U.S. Circuit Court of Appeals' recent decision regarding SCM Corp., the corporation best known for Smith Corona typewriters, which failed in its effort to defeat a hostile takeover bid by Hanson Trust PLC.

SCM lost its battle with Hanson after the Second Circuit, which sits in New York, ruled that one of SCM's most important anti-takeover maneuvers was illegal, even though SCM directors had adopted it after following the advice of some of New York's top legal and financial advisers.

"For better or worse, the courts are now recognizing that, to some extent, takeovers are different from the body of 'business judgment law' that grew up in a less-charged atmosphere," said Wilmington merger lawyer A. Gilchrist Sparks III. "In takeover situations, the courts are going to impose some additional burdens on directors."

"I think it is appropriate that managements have a heavier burden to bear under the business-judgment rule in takeovers than when making conventional business decisions," SEC Chairman John S. R. Shad said.

Meanwhile, as the courts get more aggressive, Congress remains an observer. Congress and the SEC, reflecting the free-market policies of the Reagan administration, have refrained from taking significant action on takeovers.

The new, more aggressive attitude of the courts is exemplified in the Hanson-SCM case decided in New York on Jan. 6. A review of that takeover battle and the court's ruling reveals just how far judges are willing to go in challenging business decisions by directors, especially when it appears that the directors may not be acting in the best interest of stockholders. Hanson Trust v. SCM Corp.

Hanson, a giant British conglomerate, has U.S. operations that include Ball Park hot dogs and Endicott shoes. The company's U.S. holdings, headed by Sir Gordon White, have grown over the years through acquisitions.

According to the Second Circuit majority opinion in the case, here is what happened as Hanson waged its battle for control of SCM:

On Aug. 21, 1985, Hanson announced its intention to make a $60-a-share cash offer for any and all shares of SCM, a company headed by Paul H. Elicker. Hanson made this offer directly to SCM's stockholders, because Elicker and the SCM board of directors opposed the takeover. (Hanson's bid was called "hostile" in the financial press because management of the target company, SCM, opposed the offer.)

On Aug. 22, SCM management met with investment bankers from Goldman, Sachs & Co. and legal advisers from Wachtell, Lipton, Rosen & Katz to discuss a response to Hanson's bid. Among the alternatives considered to defeat Hanson's bid was a management buyout, in which members of SCM management and outside investors would make a competing bid to buy SCM. By Aug. 23 or 24, SCM management and Goldman Sachs had initiated discussions with two firms that might help SCM management finance a buyout -- Kohlberg Kravis Roberts & Co., and Merrill Lynch.

Martin Lipton, senior partner of the Wachtell, Lipton law firm, advised the SCM board of directors on Aug. 25 that a management buyout would be the best approach. On that day, the SCM board of directors gave Goldman Sachs and Wachtell Lipton the green light to contact firms that might be interested in buying SCM. By Aug. 30, the advisers reported that none of more than 40 companies was interested in acting as a "white knight" in a friendly merger with SCM, and that only Merrill Lynch was interested in participating in a management buyout. Meanwhile, Hanson continued to say it would pursue its bid to acquire SCM. SCM did not respond to Hanson's requests for a meeting to discuss its offer.

Following five days of negotiations, SCM management, Goldman Sachs and Merrill Lynch reached an agreement for Merrill to pay $70 a share for SCM. This deal, which still needed the approval of SCM directors, also would give SCM management the special right to own up to 15 percent of the company after the takeover.

Merrill, eager to be compensated for its time and not wishing to be used solely to initiate a bidding war for SCM, was granted an immediate $1.5 million fee known as a "hello fee," and a $9 million "breakup" fee known as a "goodbye fee," the latter to be paid in the event that a third party such as Hanson acquired enough SCM shares to block SCM's deal with Merrill.

On Aug. 30., via telephone, the SCM directors voted to accept Merrill's $70-a-share offer. The directors did not have any documents describing the deal until after they had voted unanimously to approve the transaction. The terms of the proposal were described to the directors in detail over the telephone, just before they voted their approval.

In response to Merrill's $70 offer, Hanson announced on Sept. 3 that it would raise its offer for SCM to $72 a share in cash. Hanson again unsuccessfully tried to persuade SCM management to meet to discuss a "friendly merger."

On Sept. 10, Merrill agreed to increase its offer to $74 a share. In return for Merrill's agreeing to raise its offer to top Hanson, SCM agreed to place Merrill's $9 million "goodbye fee" into an escrow account, and paid Merrill an additional $6 million "hello again" fee.

Most significant, SCM agreed to grant Merrill an option, known as a "lockup option," to buy SCM's pigment business for $350 million and its Durkee Famous Foods business for $80 million, if Hanson acquired more than one-third of SCM. Merger experts have said the true value of the two businesses is closer to $600 million, far above the $430 million Merrill agreed to pay.

SCM gave Merrill this "lockup option" for two reasons: to discourage Hanson from acquiring one-third or more of SCM, because this would trigger the option and give Merrill the right to buy SCM's valuable pigment and food businesses for prices far below their fair market value; and to ensure that Merrill's $74-a-share offer for all of SCM was the winning bid.

In a three-hour meeting that lasted until midnight on Sept. 10, SCM's board of directors was informed of the new Merrill deal, including the lock-up option. Goldman Sachs advised the SCM board that the $74 offer from Merrill was fair and was the best available. Goldman Sachs also advised the SCM board that higher prices could be obtained for SCM's food and pigment businesses if they were sold separately, rather than granted to Merrill under the lock-up option.

Lipton advised the SCM board that, without the lockup option, Merrill would not agree to go forward with its $74-a-share offer. The SCM board voted unanimously to approve the deal.

On Sept. 11, Hanson, refusing to give up, purchased about 25 percent of SCM's common stock on the open market. Hanson purchased an additional 545,000 shares of SCM stock between Oct. 2 and Oct. 4, bringing its SCM holding to about one-third.

On Oct. 8, Hanson announced that it would raise its bid one more time to beat Merrill's $74-a-share offer. Hanson said it would offer $75 a share cash for any and all shares of SCM common stock, with one condition -- Merrill had to drop its lock-up option or a court had to rule that the lockup option was invalid. Hanson sued to overturn the lockup, arguing that it was not in the best interest of SCM shareholders, who could receive $75 a share from Hanson but only $74 a share from Merrill.

A lower court judge ruled in favor of SCM, and Hanson appealed to the Second Circuit. The issue before the appellate court was whether SCM's decision to grant Merrill the lockup option was covered by the "business judgment rule," and thus protected from judicial interference, or if this move to thwart Hanson could be overturned by the court.

In reversing the lower court ruling, the Second Circuit found that SCM directors failed to excercise "due care" because they did not question their takeover advisers sufficiently about the lockup option, and because the lockup option served to retard, rather than promote, the bidding process for SCM. In a written opinion, a majority of a three-judge Second Circuit panel said:

"In evaluating the acts of SCM's directors in the present case, we remain mindful of our overriding concern . . . that the role of the court in an action to enjoin block takeover measures is to allow the forces of the free market to determine the outcome to the greatest extent possible within the bounds of the law. . . . In sum, we think the lockup option forecloses, rather than facilitates, competitive bidding."

" . . . In short, the SCM directors' paucity of information and their swiftness of decision-making strongly suggest a breach of the duty of due care . . . . The board failed to read or review carefully the various offers and agreements and instead relied on the advisers' descriptions. . . . We view the board as only minimally fulfilling, if not abdicating, its role.

" . . . We further believe that irreparable injury to the SCM stockholders, including Hanson, is at stake, and that the balance of hardship in this case tips decidedly in Hanson's favor. For if the lockup option is exercised without completion of the Merrill-SCM merger, SCM will likely be broken up for inadequate consideration . . . . This harm is not protected by the business judgment rule."

The court blocked Merrill from exercising the lockup option to buy SCM's pigment and food businesses. Merrill did not raise its offer for SCM from $74 a share. Within days, Hanson took control of SCM, paying $75 a share.

The court's decision to block the lockup option thus resolved this takeover battle in Hanson's favor. Impact of Hanson Ruling

Some lawyers believe the Second Circuit decision represents a new, broader definition of the business judgment rule, which suggests there could be further judicial interference in decisions by corporate directors. Others read the opinion narrowly, saying it focuses only on the improper use of the lockup option, or only on the court's desire to be sure shareholders get the best price for their shares once the decision has been made to sell a public company.

"This is a very, very significant decision," said Dennis Block, the merger lawyer with Weil, Gotshal & Manges who was on the winning side of the SCM case. "The New York court in SCM went out of its way to say [directors have] a heightened duty of care. When you deal with takeovers, there is a heightened sensitivity."

"My opinion is that both decisions are wrong," said New York merger lawyer Martin Lipton, who was on the losing side of this case and a similar Delaware Supreme Court case Revlon involving lockup options.

"The directors in both cases acted properly, and the business-judgment rule should have sustained their actions," Lipton said. "But I think it is clear that the courts do not like lockups, and they are going to impose a more stringent application of the business-judgment rule to lockup cases than to other cases."