Controversy over what Billy Graham did or didn't say during his visit to Moscow last week has all but eclipsed attention to the central reason for his trip, namely an international, interfaith conference on peace.

Nearly 600 people from 90 countries representing every major religion attended the conference, which was organized by the Russian Orthodox Church.

For many in this country, that sponsorship cast a shadow of suspicion that the gathering would turn out to be an exercise in propaganda for the Soviet Union.

But most of the vanguard of returning American delegates this week indicated that such suspicions were unfounded, that both the conference and the three major statements it produced reflected the concern of all religions about the threat nuclear weapons pose to all human life.

The conference documents--separate messages to religious bodies, to governments and to the United Nations--were, said Dr. William P. Thompson of the United Presbyterian Church, "remarkably well balanced--they were not just a parroting of the Soviet line."

The messages called for "an instant freeze" on all nuclear weapons "as the first step" in averting a nuclear holocaust.

They called on both the United States and the Soviet Union to step up disarmament efforts and appealed to the United Nations to "devise machinery" to facilitate that goal.

The Americans interviewed agreed that East-West political tensions were present at the conference, but most felt that such conflict had been blown out of proportion in some press accounts of the gathering.

Peter Dyck, veteran Soviet expert of the Mennonite Central Committee, said that "there was a minimum of political rhetoric at the sessions.

"Once in a while there was an outburst of some individual, but these people were immediately called to order."

Reinhold Kerstein of the Baptist World Alliance recalled one such incident when "the representative from Syria was given two minutes to bring greetings, and it developed into a 40-minute anti-Zionist, anti-American imperialism harangue."

Bishop David Preus of the American Lutheran Church was chairing the session that day, Kerstein recalled.

"He made a strong appeal not to make this conference a form of East-West confrontation with mutual attacks, putting participants into the position of having to answer to their governments when they got home as to why they had not defended their own countries," Kerstein said.

The next speaker on the program, scheduled weeks before, was the Rev. Arie Brouwer, general secretary of the Reformed Church in America.

In his address he said American churches were concerned about the nuclear policy of their own government but that "we also lament the participation in the arms race by the Soviet Union and other countries."

While the conference did not follow Western parliamentary procedure, Thompson said he felt the procedures were fair.

As an example, he explained that planners of the conference came prepared with drafts of the statements that were to be issued as the final documents, but they also provided for a committee that would refine the documents for the full body to approve or disapprove.

When that committee--which included two Americans--met, he said, "they promptly tossed out the documents that had been prepared and began from scratch."

The committee drafts were then submitted to the full body and "anybody could say anything about them they wanted to--and they did."

Thompson, who has long been a leader in interdenominational and interfaith affairs, said the conference was "an important chapter in continuing relationships between religious bodies in all parts of the world . . . .

"It was successful," he continued, "and any time that kind of a mix is successful, the result is an increase in understanding."

For Dyck, who until his retirement last year handled liaison with Eastern Europe for the Mennonite Central Committee, the major achievement of the conference was "that it took place--that they met."

Dyck, who spent the first 12 years of his life in Russia and who lost "most of my relatives" in the Stalinist purges of the '30s, said there was agreement at the meeting that "the world climate now is pervaded by distrust and fear and suspicion. . . . That point was made by the East as well as the West."

The religious leaders agreed, he said, "We can't leave peace-making up to the governments. It's too important."

But the Rev. Paul L. Brindjar of the Lutheran Church in America, who had been skeptical about the conference from the outset, returned with his skepticism intact.

He felt that the concern of religious leaders over the nuclear threat was "frustrated by the controlled nature of the conference," he told the Associated Press.

Thompson said a delegation of six Americans, including the heads of four denominations, took time off from the conference to visit the six Pentecostals who are living in the American Embassy while they seek permission to emigrate to the United States.

Then, accompanied by the general secretary of the recognized Baptist church in the Soviet Union, "we talked to a government official about their plight."

He said the official was "very sympathetic" and promised the Americans that "he would express our concern" over the situation. But Thompson, a lawyer by training, was pessimistic about any imminent solution to the problem. "I think the two countries have gotten themselves into loggerheads over this issue," he said.