Member of the strife-torn United Mine Workers turned out in heavy numbers yesterday to decide an acrimonious three-way leadership fight that could have a major impact on the union's future and the nation's energy production.

The first inconclusive returns showed Arnold R. Miller, the incumbent president, running ahead of his two challengers, UMW Secretary-Treasurer Harry Patrick, and the Lee Roy Patterson, a leader of the old-guard faction ousted by a reform team headed by Miller and Patrick five years ago.

At least one apparent case of vote fraud was reported when a ballot box in Mt. Carmel, Pa., was found to contain 200 Patterson votes before the poll opened.

About 277,000 miners are eligible to vote and an official count will not be completed before July 15.

Yesterday's voting in 850 UMW locals scattered across the United States and Canada followed a bitter campaign staged against a backdrop of wildcat strikes and other signs of dissension in the once powerful fiefdom of the late John L. Lewis.

Both government and industry officials nervously monitored the race, fearful that an inconclusive or contested outcome could lead to a nation-wide strike when the present three-year UMW contract expires Dec. 6, if not before.

Concern has been particularly acute because a strike would coincide with the Carter administration's new energy policy pegged to a massive increase in coal production: from 650 million tons to 1 billion tons annually by 1985.

Patterson and Patrick blamed the union's difficulties on Miller's five-year stewardship, which they described as inept. Miller said the blame lies with those who refused to cooperate with him, such as Patterson and Patrick.

Miller, a 54-year-old West Virginia miner and black lung victim, rode to victory in 1972 on a tide of reformers' zeal following the murder of union insurgent Joseph A. (Jock) Yablonski. He defeated W. A. (Tony) Boyle, who was subsequently convicted of ordering Yablonski's murder, although a new trial has been ordered.

Patrick, 46, also a West Virginia miner, was elected with Miller on the "Miners for Democracy" slate, but split with him after a series of disagreements. Patterson, 42, a former Boyle supporter and leader of the anti-Miller majority faction within the UMW's executive board.

Patterson, charging campaign irregularities by Miller and Patrick, vowed during the campaign to challenge the election if he should lose. The heavivly pro-Patterson executive board has the power to overturn an election and order a new one.