D.C. Mayor Marion Barry, himself an expectant father, told a Father's Day Audience at Vermont Avenue Baptist Church yesterday that, like father struggling in difficult times, mayors need support, too.

His call for unity rather than criticism of his administration echoed many speeches that Barry has given to other groups in recent months, speeches designed to take Barry's view of the city's current financial crisis to the public directly, rather than through an often critical press.

"We need to pull together as a community," Barry told the congregation that listened and sometimes clapped politely, but not enthusiastically. "It's our problem, not just the mayor's problem or the city council's problem. You should be critical, but not divisive . . . not frustrated."

The Rev. John R. Wheeler, pastor of the chruch at 1630 Vermont Ave. NW, one of the city's largest Baptist churches, said he asked Barry to speak on Father's Day because he felt the mayor would project a strong black male image that would be an inspiration to the church's young people.

"Barry is our leader," said Wheeler, who became one of the mayor's strongest backers after supporting former City Council chairman Sterling Tucker in the Democratic mayoral primary two years ago.

"He's a good leader, as far as I'm concerned, and I'm proud of him," Wheeler told his congregation. "We want him to come here and and say whatever he wants: just his presence here is an inspiration."

A man in a blue leisture suit stood up in the balcony and took a snapshot as Barry was introduced. "He looks a lot thinner than I expected," said one woman, whispering to her neighbor. Women in Sunday hats and teen-age girls leanded forward in their pews, some smiling, others saying "Amen," when Wheeler asked "Mrs. Mayor," Barry's wife, effi, to stand. The Barrys are expecting their first child any day now.

As the television camera lights glared from the balcony and the choir sang, "There have been times I did not know right from wrong . . . times I felt so all alone." Barry then spoke the refrain: Through it all, I learned to trust in Jesus."

A short while later, Barry rose from the large, ornate pulpit chair to praise Wheeler ("You have a dynamic leader in Rev. Wheeler), then the congregation ("A church that prays together stays together").He said he was proud to stand in the spot where struggles have been carried on."

His was a message that linked the difficulties that black families have faced through slavery, unemployment, drug abuse and inadequate housing to the difficulties he faces a mayor of a largely black city.

"The welfare system encourages the father to leave the house, and we have to reform the welfare system so that it will not have to be that way," Barry said.

There are 5,000 Washington, D.C. men in jail today, 95 percent of them balck. Some of them ought to be there, but there are too many who, if they had been given the love of a father earlier in life, wouldn't be there today," he said. "When we think about Father's Day, we have to reach out and touch someone and say, 'I love you.' I hurt when your hurt, I suffer when you suffer."

Barry then began to explain the city's budget deficit and how it arrived at its financial dilemma. "Nothing is more agonizing than to say to someone in need that you cannot have something. No way can you go home at night and sleep well knowing that you have to lay somebody off."

Explaining that Congress was responsible for retirement payments to its fire and police departments and public school teachers, Barry said Congress should shoulder its responsibility and "carry their fair share."

He spoke of the need for jobs for youth and the apparent unwillingness of private enterprise to provide more summer jobs. "When young people aren't able to make [money], they are going to take it," Barry warned.

At a time when the average cost of a D.C. home is $90,000 or more -- an expense he said most black families can't afford -- Barry told the congregation, "If you own a house, don't sell it."

"If you have to sell it, sell it to someone in your family," a plea he stressed strongly at a time when many middle-class black families, the voting strength among blacks in the city, have begun to abandon the District for the less expensive suburbs and for what they believe are better schools.

Barry's plea for unity apparently reached many in the congregation. But some of those interviewed have already moved to the suburbs and say they are unable to help much, even those who work in the city.

"He pointed out the problems in an interesting way, but I guess the most significant thing about his speech was that people of the District need to be united during rough times," said John Mills Jr., a Seat Pleasant resident who was born in D.C.

Dorothea Williams, a Takoma Park resident and church member, said, "I feel his hands are tied, like all of our black leaders. I feel sorry for him. fWashington is mostly balck, Congress is mostly white, so there you go."