Major shareholders in McGraw Hill, Inc. are preparing to wage a proxy battle to wrest control of the giant publishing firm from a board of directors that turned down a $40-a share takeover bid from American Express Co.

Maxwell Geffen, a large shareholder in McGraw-Hill said that while a final decision to go ahead with such a fight has not been made, he has come to the conclusion "that a proxy fight probably is necessary" and said he likely would lead it.

Reached in Florida, Geffen said he was in the middle of writing a statement of his position that he would release "in a day or two."

Geffen and other major shareholders, including family member Donald C. McGraw Jr., and legal and financial advisers held a meeting in Florida last weekend to discuss possible legal moves in the wake of the flatout rejection of American Express overtures by the McGraw-Hill board last Wednesday.

American Express originally offered $34 a share for McGraw-Hill stock, but the takeover bid was hotly fought by chairman Harold W. McGraw, Jr. and other members of the publishing company's board.

Last week, American Express withdrew the $34 offer and said it would pay $40 a share, but only if the board agreed not to fight a takeover attempt and allow American Express to make the offer directly to shareholders. At $40 a share, the takeover would be worth about $1 billion. There are approximately 24.5 million shares of McGraw-Hill stock outstanding.

Donald McGraw, in a telephone interview, said he has not decided whether he would join a proxy fight, but said he would not lead it McGraw, who was a member of McGraw-Hill's board until last April, has urged his cousin Harold to stop fighting American Express and sit down and negotiate with the giant credit card and traveller's check firm

McGraw said today that he thinks the board was wrong to reject American Express' new, "friendly" bid and said he sees nothing in the board's contention that the takeover would be illegal.

"I don't want to start anything personally until I have more information," McGraw said. "If there is something illegal about it and we wage a proxy fight and win and install a new board and probably a new chief executive officer, then find there is something illegal, we could wreck the corporation and everybody could lose."

However, McGraw said, "I've been talking with my lawyer all weekend. We cannot figure out what is illegal about it. I think the McGraw-Hill board was misinformed."

He said his brother John, a board member, visited him this weekend in Florida and said John is convinced that the takeover is illegal.

John L McGraw voted with the rest of the McGraw-Hill board last week to reject the American Express offer. Both Donald and John were ousted as officers of the company by their cousin Harold.

Donald McGraw said that he still thinks McGraw-Hill should sit down and negotiate with American Express. If there is something illegal in the takeover bid, as McGraw-Hill's board alleges, "that would come out in the negotiations and there would not be the bloody war that there is now."

Geffen has said that he thinks that as a matter of corporate democracy shareholders should have the right to decide whether to take the $40 offer or not.

Geffen said today that his statements in support of corporate democracy put him in a leadership position in any proxy battle. "I guess I'll have to be it (the leader)."