Former Treasury Secretary William E. Simon last night withdrew his $12 million offer to buy the Baltimore Orioles.

"As far as I'm concerned, my offer is withdrawn," Simon told the Baltimore Sun, confirming that his decision had been relayed to Jerold C. Hoffberger, principal owner of the franchise.

"Mr. Hoffberger wants to play both ends against the middle," Simon continued. "Well, he can forget this end. I think at this point and at this time the game is over. He has damaged the merchandise and acted in bad faith. I think I've been played dirtly pool everywhere to Sunday."

Simon had been considered the front runner of the various parties interested in purchasing the Orioles. He began attempts to buy the American League baseball team last summer.

His trying to buy the club prompted a group of Baltimore businessmen to try to raise matching funds to buy the club to ward off what they believed would inevitably be a shift of the franchise to Washington if Simon were successful.

Hoffberger told the Sun last night that he regretted the development because "Mr. Simon is a gentleman. He would have been an asset to baseball as would have been his partner." Simon's partner has never been identified.

Hoffberger told the newspaper he intends to arrange a low-interest loan to the Baltimore group through a company or group controlled by the Hoffberger family.

The Sun reported that the loan would be about $4 million to be repaid at 6 percent interest in the next five years.

Hoffberger said that several other people have made inquiries about buying the club but that he considered the Baltimore group foremost.

Under the arrangement, Hoffberger would remain a prime partner. It was not immediately clear last night if Hoffberger had lowered his price for the group, which reportedly had been able to raise only $6 million on its own and was contemplating a state or city loan if available.

A member of the Baltimore group told The Washington Post last night that last night's developments mean there will be no attempt to seek state aid. He said Simon's first offer was "not up to the dollar price (Hoffberger) wanted" and that Hoffberger's future role probably would be "an advisory one," but that the issue was "not clear yet."