SOON AFTER JOSEPH Yeldell entered his suite in the National Theater Building on Pennsylvania Avenue where he worked until last week as the mayor's general assistant and political advisor, a secretary looked into his office and said, "Mr. Yeldell, the mayor on line one."

Yeldell spun around in his padded swivel chair and punched in to Walter E. Washington on his telephone console. "Good morning, Mr. Mayor," Yeldell said with the businesslike formality he uses with his friend when they are being observed. "What can I do for you today?"

After listening for a while, Yeldell stroked his chin, pursed his lips, leaned back in his chair and smiled. "Ummm, let me see now . . .

"Actually, I think we should try it the other way, Mr. Mayor. It's a mine field out there. Both of us know they'll try to cut you to shreds. Yeah, yep, yep. That's right, Mr. Mayor. They are just waiting for you. But I'll take care of it right away, sir. You know you can count on me."

Yeldell would not say what that call was about but it nevertheless did show something about his influential relationship with the mayor. A while later that morning, Yeldell's secretary told him that another telephone caller wanted to know what the "P" in Joseph P. Yeldell stood for.

"Powerful," Yeldell said.

That Joseph Philip Yeldell could boast even as he knew that a federal grand jury was probably only days away from indicting him for bribery was as much a measure of his remarkable personal resiliency as his enduring friendship with Walter Washington.

During nearly 11 years in local politics, Yeldell had been on a roller coaster ride, soaring high at some points only to drop nearly out of sight and then suddenly to reappear on the rise again.

After serving on the city council, losing an election for nonvoting D.C. delegate to Congress, spending five stormy years as director of the city's mammoth Department of Human Resources, being removed from that job amidst charges of nepotism, cronyism and corruption, he had reappeared once again even nearer the top as the number three official in the Walter Washington administration - general assistant to the mayor with spacious offices across Pennsylvania Avenue from the District Building. Power Gone for Now

Yeldell is now on leave from that job, sent home by Mayor Washington last week after the federal grand jury indicted Yeldell and millionaire businessman Dominic F. Antonelli Jr. on charges of bribery. The power for which Yeldell has been grasping for so long is again, at least for now, gone from his hands.

He had first been introduced to power in 1967 after being appointed by President Lyndon B. Johnson as the Anacostia representative on the newly formed D.C. city council. He quickly became chairman of the council's personnel committee, putting himself in charge of hiring his colleague's secretaries and office staff. With that power, Yeldell began building himself a constitutency of city employees dependent on his patronage, a core of support that would remain with him to this day.

"He always seem to know what was important, politically," recalled Stan Anderson, who also was a member of the 1967 city council. "He seemed to know by instinct how to maneuver into the rights committees, things like that."

While on the city council, Yeldell cultivated the new mayor, Walter Washington, who became Yeldell's friend, Yeldell supported the mayor unsweveringly, gossiped with him about secret city council strategies and, in time, became known on the council as "The Mayor's Man."

When the mayor needed council support and community understanding about a political hot potato, the location of city incinerator No. 3, the mayor went to Yeldell. At issue was a huge sanitation department smoke-stack, out of which thick gray and black smoke would spew - The two sites offered to the mayor were a vacant lot in Georgetown and a vacant lot next to a public housing project in far Northeast Washington. With Joe's support, the decision was made to have the smoke rise over the Kenilworth Courts Housing Projects, right through the high intensity power lines that the government later located nearby on the other side of public housing projects.

Yeldell the politician was still able to glad hand his way through the neighborhoods and schools of low-income areas where few politicians had dared to venture before. Talking in Streets

During the 1968 riots, Yeldell and fellow council member Anderson were often seen in the streets, talking with youths about the problems of inner city life. When one street tough was about to throw a punch at Sen. Charles Mc C. Mathias (R-Md.), who was surveying damage and interviewing ghetto residents, Yeldell stepped between the youth and senator. "Be cool, man," Yeldell warned, and he was cool.

By 1971, in the middle of his second term as councilman, Yeldell went after what seemed to be more power as the city's first nonvoting delegate to Congress. Through his cautious, conservative and businesslike approach on the city council, Yeldell had picked up considerable support in Washington's business community.

With the financial backing of the same upper Northwest Washington elite that has stunned when he was first appointed to the council, Yeldell waged a conservative campaign, emphasizing fiscal responsibility and non-violence in opposition to the still trident rhetoric elsewhere in 1971.

The poor boy from Cardozo High was now receiving campaign contributions from the likes of Thomas Owen, president of Perpetual Savings and Loan, and parking and real estate investor Antorelli. Realtor and former D.C. Democratic national committee woman Flaxie Pinkett, who lived in the exclusive Gold Coast area of upper Northwest Washington, became

He was defeated, however, by then-council vice chairman Walter Fauntroy, his old archrival as a high school debater who was viewed by many as Yeldell's cocampaign manager. Yeldell also had the backing of the mayor. the more progressive candidate for nonvoting delegate. Offer From Mayor

Though he lost the election, Yeldell would months later receive from his friend, the mayor, an offer he could not refuse: control of the city's newly created Department of Human Resources, the immense 10,000-employe social service agency that spends about 20 percent of the city's $1 billion budget.DHR touches the lives of nearly one of every five city residents, particularly those with critical needs, through its control over the welfare payments, community mental health centers, food stamps, medicaid, medicare, emergency food service, the city-run student loan program and, at the time Yeldell took over, D.C. General Hospital, the city's only public hospital.

The mayor had consolidated numerous smaller programs into the mammoth agency only months before Yeldell took charge in December 1971 in an effort to bring order to several entrenched and chaotic health and welfare fiefdoms which had been ruled for decades by white bureaucrats who had believed they alone knew what was best for poor people. That thinking had clashed with the activism of the 1960s, when welfare recipients began, through protests and court suits, to demand that their rights as citizens be honored and their voices heard. The health and welfare programs themselves were critcized as being harmful to many of those they served.

This was what was inherited by Yeldell. It was a big jump for Yeldell who had been a teacher, an IBM salesman and a city council member, but had never before been given day-to-day management over any organization larger than a city council committee or a math class at Collidge High.

In an interview a few weeks ago, Yeldell recalled, "I was somewhat perplexed, confused at first about why the mayor picked me. BUt I couldn't turn it down. Blacks just didn't get those kinds of management opportunities every day."

Yeldell forced out the cadres of bureaucrats who believed in the old, paternalistics approach to social work. In doing so, Yeldell aroused resentment and he knew it. He often complained of social workers being out to get him, of bureaucrats trying to sabotage his programs. Purges Cheered

Many of the old social service agencies' critics cheered his purges as long overdue. But, in time, they would become concerned, then angry that Yeldell failed to bring in persons to help him who could strongly and effectively administer the unwieldy agency. The loyalty of his lieutenants, his critics would say, instead became his overriding priority.

Yeldell brought into DHR management Joseph Douglas and Lemmie Morton, his old Phi Beta Sigma fraternity brothers at D.C. Teachers, as deputy director and chief of welfare inspections.

In late 1976, The Washington Star disclosed that four members of he Yeldell family also had shown up on DHR payrolls. In addition, the daughter and son-in-law of Cleveland Denington Technical Institute got jobs at and one of his brothers had jobs at the Washington Technical Institute.

In addition, Yeldell tended, his critics and even sympathetic observers said, to operate the department by imposing the sheer force of his will on a situation and falling back the aboslute certainty that he was always right, is failure to delegate authority broadly and his tendency to become preoccupied with the crisis of the moment were constantly mentioned by critics and other observers.

By late 1976, nearly every major institution and program in Yeldell's department had been criticized, investigated or threatened with the loss of federal funds. D.C. General Hospital had losts its accreditation and D.C. Village, the city facility for the elderly, had to stop admissions because it could not hire sufficient nurses.

Yeldell also drew fire for the ways he used government money. Sen. Lawton Chiles (D-Fla.), then chairman of the Senate District appropriations subcommittee, alleged a "pattern of misues" against Yeldell's department, including diversion of $8,740 in maternity funds to rent air-conditioned cars, its expenditure of $30,000 in federal funds to refurbish the office of the DHR welfare chief and its order to build a bathroom and a kitchen in Yeldell's already spacious and well-appointed DHR office.