Mayor Marion Barry declined to take a stand yesterday on D.C. Del. Walter E. Fauntroy's controversial Middle East peace campaign, but he did say he didn't believe Fauntroy's efforts would hurt the drive for full Congressional voting rights for the District.

"Any American citizen, as long as they are in the legitimate boundaries of law, ought to be able to go anywhere they want to go . . . You can go anywhere you want to go as long as you can afford it," Barry said on yesterday's "Morning Break" television show (WDVM-TV 9).

It was the closest that Barry has come to speaking out on the issue after nearly a week of silence. Until yesterday, Barry has repeatedly refused to either endorse or criticize Fauntroy, whose latest efforts included a visit to Lebanon last week and an invitation to Palestine Liberation Organization leader Yasser Arafat to speak in the United States.

An interviewer asked the mayor if he had problems with Fauntroy meeting with representatives of the Palestine Liberation Organization and Arafat, its leader. Barry responded, "I don't have any problems with anybody."

Later in the day, Barry's office said that the remarks did not constitute an implicit endorsement of Fauntroy's actions. "That statement cannot be interpreted as anything other than a statement by the mayor," said spokesman Kwame Holman.

Barry was explicit when asked on the television show if the controversy surrounding Fauntroy's trip -- Jewish leaders have been especially upset by the invitation to Arafat -- could result in the loss of Jewish support for the congressional voting rights drive.

"I think that voting rights is going to stand on its own," Barry said. "Anyone who does not support our freedom and independence based around some disagreement in the Middle East wasn't going to support it any way."

Barry acknowledged that there were "concerns" among blacks and Jews in the city about the effect that Fauntroy's mission might have on the two groups, who Barry said had "struggled together" over the years. But the mayor accused the news media of exacerbating the situation and asked for an end to the discussion of a black-Jewish rift.

"Too much has been said about this. There are a number of issues facing our city and our country," Barry said. "I'd rather talk about jobs and education and ANCs (Advisory Neighborhood Commission) and community development and other things."

The tone of Barry's remarks were in line with growing call for moderation on the issue by both Jewish and black leaders.

At the heart of the controversy is the effort of Fauntroy, in his capacity as chairman of the Southern Christian Leadership Conference, to establish a Palestinian-Israeli dialogue that Fauntroy believes is essential to a Middle East peace.

Israel and its supporters oppose any contact with the PLO because the group does not recognize the right of Israel to exist. The U.S. government essentially supports that position.

When Fauntroy announced last week that the SCLC had invited Arafat to speak at an educational forum in the United States, leaders of Washington's Jewish community said they felt angered and betrayed and thought Fauntroy was not properly representing his Jewish constituents.

Some black churchmen and Arab-American leaders in the city defended Fauntroy's peace efforts, with several saying that his critics were holding him up to a double standard based on race.

Yesterday, the Human Rights Executive Forum, a 23-member coalition of Washington area civic, religious, civil rights and human relations groups, met to discuss tensions between blacks and jews in the city.

After the meeting, the group issued a statement saying, "We sense there is a problem in human relations and we will do everything we can, using the good offices of the 23 organizations involved, to remind ourselves how much we mean to each other."