GAF Corp. is mulling a "business combination" with Union Carbide, according to documents filed by the company with the Securities and Exchange Commission.

GAF now owns more than 5 percent of its fellow chemical company, and the disclosure has prompted rumors of a possible takeover. The Wayne, N.J., firm is a fraction of the size of Union Carbide, but David and Goliath stories are nothing new there.

Stockholder Samuel Heyman won control of GAF two years ago in a fight begun after he became disgusted with the $24 million operating loss it posted in 1981. He was convinced that poor management was contributing to the company's problems, he said at the time. Owning only about 3 percent of the company in late 1981, he bought more stock and launched and won a bitter proxy battle. In December 1983, he finally wrested power from Jesse Werner, who had led the company since it went public in 1965.

"I wouldn't put it past them to wage a proxy war with other shareholders" to gain control of Union Carbide, said Tom Au, an analyst who tracks GAF for the Value Line Investment Survey in New York City. GAF now owns 5.6 percent, or 3.94 million shares, of Union Carbide, according to its SEC filing on Wednesday. The shares were bought on the open market for $192 million over the past three months, according to published reports about the acquisition.

Au said that GAF would not be able to wage a takeover of Union Carbide by itself. "They can raise some money, but not an awful lot. Not enough to take over Union Carbide. It would be through an insurgency of the kind that brought the insurgents to GAF," he said.

Union Carbide, with revenue of $9.51 billion last year, has been under the shadow of litigation since a poisonous gas leak at its plant in Bhopal, India, killed 2,000 people last December, although its stock recently has rebounded. An unexplained gas leak at one of its plants in West Virginia injured 135 people on Sunday. Yesterday, Union Carbide closed at 52, up 1/4 on volume of 70,490 shares.

Heyman has said that his prime motivation for the GAF takeover was a conviction he could help make the company more profitable, even though he was a lawyer and real estate investor with no corporate experience. Although Heyman claimed he had no interest in running the company and planned to sell many of its assets, he changed his mind once he stepped into office.

In the last quarter of Werner's leadership at the end of 1983, GAF lost $15.6 million. Last year, net income totaled $56.7 million ($3.70 a share) on revenue of $731.3 million, compared with a net loss in 1983 of $3.8 million (47 cents) on revenue of $699.4 million. Most of GAF's business is now split almost equally between its chemicals branch and its roofing branch.

Heyman cut overhead, moving the firm from its Manhattan offices to Wayne, N.J. According to the company, during the first nine months of his tenure as chairman, Heyman slashed operating expenses 23 percent, closed three roofing plants, sold off 13 building supply centers and fired 700 of 4,700 employes. Meanwhile, he cut his own salary in half, down to $250,000.

"He runs the company very tight," the spokesman said.

GAF traces its roots back to the first photo shop in America in 1842, but it got out of the consumer photo line in 1977 because of heavy competition from Eastman Kodak Co. and other companies.

In its second quarter this year, sales dropped to $188.1 million from $191.3 million in the same quarter a year ago, but net income climbed to $23.8 million ($1.35) from $15.1 million (99 cents).

The original New York City photo shop was known as Anthony & Scovill, then Ansco. In 1929, it was incorporated into General Aniline Works by the German chemical firm, I. G. Farbenindustrie, and was renamed General Aniline & Film Co. in 1939. That name made headlines in 1942 when the firm was seized by the U.S. government under the Trading With the Enemy Act. After the seizure, the government ran the company, selling photo products under the Ansco label until it was sold to the public through a share offering in 1965 in an out-of-court settlement, becoming GAF Corp.