United States Steel Corp. and Texas Oil & Gas Corp. are holding talks that could lead to a merger of the two companies, spokesmen for both organizations announced yesterday.

The talks, exploring what both sides called "a possible business combination," also might yield a joint venture in which the two companies would share the costs and profits of a new organization while retaining their individual identities.

But the betting on Wall Street is that the goal is a merger, and by one estimate, it would cost U.S. Steel nearly $6 billion to acquire Texas Oil & Gas, one of the nation's largest independent natural gas companies.

Pittsburgh-based U.S. Steel, the nation's largest steel maker, has been diversifying out of the troubled steel industry since 1981, when it paid $6.5 billion to buy Marathon Oil Co.

U.S. Steel's oil and gas businesses accounted for $5.2 billion, or 54.2 percent, of its $9.6 billion in sales for the first half of 1985. By comparison, steel-generated revenue amounted to $3.4 billion, or 35.4 percent. Chemicals, manufacturing, engineering, domestic transportation and other businesses produced $1 billion, 10.4 percent of U.S. Steel's sales for the period.

Dallas-based Texas Oil & Gas would open another road for U.S. Steel's diversification efforts, some metals industry and oil industry analysts said yesterday. But they pointed out that that the oil and gas industry also is in bad shape.

The steel industry is in far more critical condition, however, having lost $5.8 billion since 1981, its last profitable year. Texas Oil & Gas, one of the healthiest companies in its industry, more than likely will demand a premium price for its assets in a merger with U.S. Steel.

Robert E. Phaneuf, an oil industry analyst with Kidder, Peabody & Co. Inc., estimated that U.S. Steel could wind up paying $3 a share more than what his company sees as Texas Oil & Gas's current asset value of $17 a share. Texas Oil & Gas has 210 million shares outstanding. Combined with $700 million in Texas Oil & Gas debt, that means U.S. Steel may have to pay $5.9 billion for the natural gas company, Phaneuf said.

The stockholders of either company might not like that deal, Phaneuf said. To holders of U.S. Steel shares, such a price might be too costly, particularly in view of the overcapacity and soft prices currently affecting the oil and gas industries.

Though $5.9 billion may seem enticing, it is unlikely that U.S. Steel would be able to put up that amount in cash, causing it to have to borrow and issue stock at a time when its stock price is falling. U.S. Steel closed yesterday at 28, down 1, while Texas Oil & Gas stock closed yesterday at 19, down 3/8.

Spokesmen at U.S. Steel and at Texas Oil & Gas declined comment on the speculation.

According to its preliminary estimates, Texas Oil & Gas will report earnings of $277 million ($1.32 a share) for its 1985 fiscal year, which ended Aug. 31. That is 20 percent less than the $346.2 million ($1.65) the company reported in fiscal 1984.

Texas Oil & Gas said that its assets reached a new high of $3.8 billion in the latest fiscal year.

The natural gas company's continued, albeit diminished, profitability in a slumping market makes it attractive to investors, oil industry analysts said.

"There is no reason why the management of Texas Oil & Gas has to sell. If they do something like that, it'll probably be to capitalize on their favorable position," Phaneuf said.

A spokesman for the American Iron and Steel Institute, which represents the nation's major steel makers, said that U.S. Steel's diversification efforts do not mean that all steel makers will use that route to escape further losses.

Ted Taylor, spokesman for Ohio-based Armco Inc., agreed. "We have almost come full circle," Taylor said.

"In the 1960s and '70s, our theory was that we needed to diversify into businesses that would protect us from the cyclical nature of the steel industry," Taylor said. "We put a lot of our eggs into the diversification basket, but everything that we had" in new businesses "went to pot all at once" during the last recession.

Since 1984, Armco has shed nearly $800 million in assets. "We are back to being a steel company," Taylor said.