Warner Communications Inc. said yesterday it has exercised its option to buy out American Express Co.'s 50 percent share of Warner Amex Cable Communications Inc. for $450 million.

The announcement effectively blocks the bid by Time Inc. and Tele-Communications Inc. to buy Warner Amex for $900 million in cash and the assumption of $500 million of the company's debt.

Warner said it will operate Warner Amex as a wholly owned subsidiary, and announced separate plans for considering two offers to sell Warner Amex's share of MTV Networks Inc. for about $310 million.

American Express offered last month to buy out Warner's half of the jointly owned cable company. Under terms of the Warner Amex partnership agreement, Warner had to either sell its share to American Express or buy out American Express.

If Warner had decided to sell its share, American Express had announced plans to buy it and then sell all of Warner Amex to Time Inc. and Tele-Communications Inc.

American Express has publicly expressed a desire to get out of the joint venture for some time. Warner Amex, the nation's sixth-largest cable-television network, has lost money for years but is expected to break even this year. It suffered a $94 million pretax loss last year and reported a net loss of $25 million on revenue of $473 million.

"We are very pleased that the process of selling our interest in Warner Amex is proceeding in an orderly fashion under our partnership agreement with Warner Communications," American Express Chairman James D. Robinson III said after Warner's announcement yesterday.

Tele-Communications President John Malone said, "We're disappointed, but we can understand that Warner would like to stay in the business." Time Inc. executives could not be reached for comment.

Time Inc. and Tele-Communications had raised their bid twice, from a starting offer of $750 million and the assumption of $550 million in debt. As to the possibility of future offers for Warner Amex, Malone said, "You never can tell."

Warner said the decision was approved unanimously by its board of directors and that the transaction should be completed sometime next year. Warner Chairman Steven J. Ross said the decision comes "as the cable industry enters an era of prosperity and profitability."

"Ross suffered through all those years of losses, and wants to be there now that it's turning around," said Fred Anschel, an analyst with Dean Witter Reynolds Inc. The company was called Warner Cable Co. until December 1979, when American Express bought half for $175 million.

The decision is "a good investment in the long term" for Warner, Anschel said, adding that the acquisition is unlikely to affect Warner's stock in the short term. Warner's stock closed down 1/4 yesterday at 29 7/8.

The purchase also will add enough debt to reduce the chances of a takeover attempt by disgruntled board members or outsiders, Anschel added. Ross has had a widely reported falling out with Chris-Craft Industries Inc., the white knight that rescued Warner last year from a hostile takeover attempt by media baron Rupert Murdoch. Since then, Chris-Craft has boosted its stake in Warner to about 30 percent, has tried to increase its representation on the board, and has sparked rumors of a possible proxy contest.

Warner's purchase of American Express's share in the cable company will be financed by a consortium of banks, led by Bank of Boston, Warner said. Warner Amex will continue to operate in its current form, as a wholly owned subsidiary of Warner, a spokesman said.

Separately, Warner said it had entered into an agreement with Viacom International Inc. giving Warner the right to sell Warner Amex's share of MTV Networks to Viacom for $310 million, within 90 days.

Warner Amex owns two-thirds of MTV, known for its highly popular music video cable programming. One third of the company's stock is publicly owned.

Warner said it is "studying" a proposal by Forstmann Little & Co. to buy all of MTV's stock for $470 million, or $31 per share, through a leveraged buyout. In a leveraged buyout, a company is acquired largely with borrowed funds that are repaid through the sale of the company's assets or operating revenue.

Warner Amex would gain about $313 million through the leveraged buyout, arranged by Forstmann Little & Co., a privately owned investment firm, and members of MTV management.

If Warner exercises its option to sell to Viacom, the terms of their agreement give Viacom the right to acquire 50 percent of Warner Amex for $400 million in cash. That acquisition would include part of MTV, but would exclude Warner Amex's 19 percent stake in Showtime/The Movie Channel.

MTV Networks reported a 1984 profit of $11.9 million on revenue of $109.5 million. The company operates two 24-hour cable music video stations, MTV and VH-1, and a children's channel, Nickelodeon.