West Germany's radical Greens party has failed to resolve a bitter internal split between pragmatists urging cooperation with leftist Social Democrats and purists insisting on avoidance of any links with the political establishment.

The upstart party dodged a possibly fatal confrontation over the issue by approving a compromise during a special three-day convention in Hamburg over the weekend.

The resolution declared that the opposition Social Democrats are not presently acceptable as coalition partners but put off a final determination until the 1987 national elections. The Greens also agreed that local and state branches could explore forms of cooperation with other parties.

The iconoclastic Greens, who advocate drastic environmental measures and West Germany's withdrawal from the North Atlantic Treaty Organization, have emerged as the third strongest party by luring protest votes from people upset over dying forests, funding scandals and weak leadership in the political establishment.

Recent polls give them between 10 percent and 15 percent of the popular vote, a substantial rise from the 6 percent they won in the March 1983 election.

In some university towns, the Greens have even surpassed the Social Democrats to become the second largest party behind the ruling Christian Democrats.

Ever since the party's creation little more than four years ago, the Greens have been perceived as a logical partner of the Social Democrats, whose chairman, Willy Brandt, envisions the road back to power for his party through "a majority of the left."

Such "red-green" coalitions have foundered in the past when tried on an informal basis in state governments in Hamburg and Hesse.

But further attempts are likely if the Social Democrats fail to win governing majorities in the states of Saarland and North-Rhein Westphalia next year.

Successful alliances between the Greens and Social Democrats in these states would serve as important precedents for a national coalition in 1987.

That prospect, however remote, troubles the center-right government of Chancellor Helmut Kohl because a continuing slide in support for the Free Democratic partners has raised the possibility that they may not win enough votes in 1987 to meet the 5 percent minimum necessary to be represented in parliament.

The Greens' sustained political strength has forced the party to confront the choice of sharing the responsibilites of power and accepting achievement of limited goals, or acting as an antiparty opposition that retains its idealism but spurns any role in government.

The pragmatists, headed by lawyer Otto Schily and fellow member of parliament Joschka Fischer, contend that the Greens already hold a stake in government through their electoral success and should use their clout to secure some of their goals, even if it means compromise with larger parties.

The Greens have demonstrated their impact on parliament by compelling other parties to emphasize the need to curtail pollution and nuclear power. The Realpolitik elements in the party say the Greens could expand their influence in the country even more by forging a coalition with the Social Democrats.

But party fundamentalists, whose loyalist core may be the most powerful faction in the Greens, argue that any cooperation with other parties would sacrifice the Greens' identity as an autonomous movement untainted by the political system. They fear that the Social Democrats would quickly absorb the Greens' voters as well as policies within any coalition.

Marxists within the Greens favor the purist approach because they see the party winning more support at the polls from people who want to register their continuing disapproval of nuclear weapons, pollution, the current structures of industrial capitalism and the presence of U.S. soldiers in West Germany.

During the weekend debates, a majority of the 800 delegates appeared to share purist suspicions about too much intimacy with the Social Democrats and the danger that the Greens could lose their soul by working too much within the parameters of the political system and not enough on the streets.

The pragmatists also object to party rules calling for a rotation of members of parliament after two years, or halfway through their terms. The motive behind this provision is to prevent the emergence of professionals or political celebrities within the party.

The realist faction claims the Greens will waste the experience of their 27 members of parliament who will be asked to leave in March. They say that their successors will need to learn all over again how best to exploit the power of their positions, a hiatus that could cost the party some of its influence in parliamentary committees.

Even some purists, like Petra Kelly, the U.S.-educated founder of the Greens, complain that the rotation system is foolish and could prove catastrophic to the party. She says she intends to complete her four-year term in parliament to carry out her disarmament work.