Clement F. Haynsworth Jr. remembers bringing the newspapers home one night nearly 10 years ago scanning their pages to read about a man he hardly recognized-himself.

"You know, my dear," he said to his wife as they discussed the latest blistering attacks on Richard Nixon's Southern nominee to the Supreme Court, "the more I read about this man, the more I've convinced he simply will not do."

Today, at 66, Haynsworth can tell that story with a smile despite the painful notoriety he gained when Senate foes shredded his judicial reputation and then rejected his court nomination. It was the most harrowing experience of his patrician life.

But Havnsworth-then, as now, chief judge of the 4th U.S. Circuit Court of Appeals based here-quietly resumed his appellate court career. Though shaken by his public battering, he was determined to place the criticisms behind him.

"I survived it," he says simply, his sof, Southern drawl showing only traces of a speech defect that occasionally causes him to stammer.

Not only has Haynsworth survived, but his record on the 4th Circuit-one of 11 appeals courts in the federal system-today wins wide praise from even liberal law professors. Although liberals say often disagree with his rulings, most praise Haysnworth as a precise, courteous conservative jurist who got "a bum rap" by the Senate 10 years ago.

"Haynsworth," says Harvard University law professor Lawrence Tribe, "has again established himself as a quiet, distinguished, moderate jurist who deserved better than he got."

Haysnworth, almost shy and self- effacing with strangers, looks very much as he did nearly a decade ago when a phone call from President Nixon propelled him into the public eye. Working again in the relative obscurity that was his before all the furor, he still has his trim, if aging physique, the lined face and plain glasses and his receding gray hair.

In two interviews, one in his chambers here and another in his hometown of Greenville, S.C., Haynsworth spoke at lenght about his Supreme Court rejection and the work of his court-whose rulings have had tremendous impact on law and life in Virginia, Maryland and three other states.

From the day the Senate voted 55-45 against his confirmation, Haynsworth said, he has always regarded his court defeat as political. But he rejected any suggestion that he was ever bitter over the incident.

"I knew there might be a problem because I was a Southerner, but I did not expect anything like what happened," he said.

"But for the time in which he was nominated, Haynsworth would have been eminently qualified to sit on the court," said Harvard's Tribe.

William VanAlstyne, a law professor at Duke University who testified in favor of Haynsworth's nomination after researching the judge's rulings, said the Senate did not focus on Haynsworth's qualifications. "I was picking up stray comments that the judge was a notorious crony of Strom Thurmond and part of the cruel and mean-minded 'Southern strategy' on Nixon's part, and none of this really registered in terms of what I'd seen or heard about him," VanAlstyne said.

Haynsworth said he did not seek the nomination and never thought of himself as part of Nixon's Southern strategy. He was not the choice of Sen. Strom Thurmond (R.S.C.) to serve on the court, but acknowledged he had "a number of friends who did have influence in the White House."

Whether part of Nixon's political maneuverings or not, Haynsworth took the brunt of the reaction to it. He was attacked as anti labor, called a "laundered segregationist" and accused of "ethical insensitibity" because he ruled in some cases in which he had a relatively small financial link through stockholdings.

Bewildered by the attacks on his judicial philosophy and ethics, Haynsworth took little comfort in the private communications he got from certain senators at the time. "They kept telling me not to think this was directed at me personally, but I was the one whose head was being pounded."

Today there is a reminder of that pounding on the wall of Haynsworth's locust wood panelled office in Greenville. A distinctive plaque honors the judge as 1969's "South Carolinian of the Year" for his "significant influence on the news."

"My contribution was entirely passive," the judge assures a visitor, displaying the dry humor he usually invokes whenever his Suprime Court fight is mentioned.

Although former Senate Majority Leader Mike Mansfield and others have since said they made a mistake in opposing Haynsworth's court nomination, Sen. Birch Bayh (D-Ind.), the man who led the fight against him, stands by his earlier criticisms.

"The senator never at any time had anything detrimental to say about Haynsworth's qualifications," said a Bayh aide. "He was just trying to move into an era where the conduct of a judge's business and judicial affairs would enhance the court."

Standford University law professor Gerald Gunther agreed. At that time a good many judges were getting loose about the kind of stocks they were holding" Gunther said. Haynsworth's rejection was pretty awkward personally but perfectly justifiable as a way of setting new court standards."

Others disagree strongly, and a former Haynsworth clerk, Larry Ritchie, said the chief judge had always been an advocate of establishing just such standards "because he was a wealthy man and had a lot of holdings.

"The criticism to which he was subjected was political and everybody knows it." huffed Bernard J. Ward, a law professor at the University of Texas in Austin. "He was very shabbily treated."

Marthan Mitchell, the late wife of former attorney general John Mithcell, complained shortly after the Haynsworth rejection tthat it was "just like the country had slapped him in the face." And Haynsworth acknowledges he did have "a sense of hurt" at the time.

"I didn't know if I was going to feel rejected or if my effectiveness as a judge would be impaired," he said. After a brief vacation that Christmas, however, "my sense of hurt pretty much healed and vanished."

Also largely healed today is the image of Haynsworth. He spends much of his time either in Richmond, where the court hears oral arguments the first week of each month, or in Greenville, where he drafts most of his opinions.

The court's sometimes controversial rulings in civil rights and labor law cases, its landmark decisions on behalf of prisoner rights and its recent reversal of the mail fraud and racketeering convictions of former Maryland Gov. Marvin Mandel and five condefendants have additionally focused attention on the 4th Circuit.

One of the oldest and smaller appelllate courts in the nation, the 4th Circuit is the intermediate stop for any case bound for the Supreme Court from the district courts in Maryland, Virginia, West Virginia, North Carolina and South Carolina.

Although some lawyers who have argued before it call the 4th Circuit one of the most conservative in the country, Haynsworth and others are uncomfortable labeling either the court or its members.

"Labels like that are very deceptive to start with," he said. "When you're a judge, one side must lose and one side must win and etiher side is going to be unhappy with you sometimes."

Alexandra civil rights lawyer Philip Hirshkop is one of the court's strongest critics. Haynsworth's opinions "make me want to pull my hair sometimes," he said, describing the chief judge's decisions as "really shallow in the rights field." But he also praised Haynsworth as a person, calling him gentlemanly and courteous.

That extreme courteousness is also part of the court's special flavor and the judges have what one lawyer called the "quaint, traditional custom" of coming down form the bench and shaking hands with the attorneys after hearings.

Those who know the court well say Haynsworth falls somewhere in the middle philosophically among his colleagues. This puts him to the left of very conservative jurists like H. Emory Widener Jr. of Virginia and Donald S. Russell of South Carolina, but the chief judge is definitely to the right of two other members of the court, Harrison S. Winter of Baltimore and John D. Butzner Jr. of Richmond.

All of the 4th Circuit's jurists, with the exception of Haynsworth and the court's newest member, J. Dickson Phillips of North Carolina, were District Court judges prior to joining the appellate panel. These include Widener, Russell, Winter, Butzner and Kenneth K. Hall of West Virginia. Herbert S. Boreman of West Vrginia, Albert V. Bryan Sr. of Alexandra, and John A. Field of West Virginia serve as senior judges.

Haynsworth, who earns $57,000 a year, must step down as chief judge when he reaches 70. But he says he is looking forward to assuming senior judge status-called semi-retirement by some-even sooner. He says he has "no intention of staying on as chief judge until I'm 70." Winter, next in seniority, would replace him.

"Historically, it's been a strong, solid court," said Daniel J. Meador, who heads the Justice Department's Office for Improvements in the Administration of Justice. "I think well of it, and it stacks up well in the quality of its judges, its opinions and the way it is administered."

Larry Ritchie, who clerked for Haynsworth in 1967-68 and now teaches at Georgetown University Law Center, particularly likes the way "you cannot stand up and read or make a speech because the judges are going to start peppering you right away with questions."

Ritchie said the alertness and inquisitiveness of the 4th Circuit contrasts sharply, for instance, with the D.C. Circuit in Washington. There, he complained, "98 per cent of the time you have a sleepy panel whose members aren't going to ask any questions at all."

The 4th Circuit, Ritchie said, "is not any more conservative than any other court in the country. It does a very workmanlike, scholarly and legal job, and it's been ahead of everyone else in the field of prisoner rights and criminal law."

Texas's Ward ranks the court "in the middle" on affirmative action cases. But he notes that the 4th Circuit's opinions aiding prisoners-"the real minority of our time"-have been very important. "And let's face it, most prisoners are black."

Ironically, the man who divided the Senate is seen as something of a mediator and cohesive force on the appellate court to which he has belonged for 22 years, 15 of them as chief judge.

Noting the court's practice of meeting in rotating three-judge panels half the year, Ward said that in such close contact "a little acerbity can ruin the show."

All the judges share a great affection for Haynsworth, Ward said "and fall all over themselves not to create any friction or faction that would disturb Clement."

"My concern is to keep an open dialogue of all the judges involved so that you don't have one or two judges getting together," Haynsworth said.

While in Richmond, the judges live at the same downtown hotel and nearly all of them dine together every night. Draft opinions from the smaller panels are circulated among all the jurists so they have a chance to comment before publication.

If anything still disturbs Haynsworth about the Supreme Court flap it is the blurring in the public's mind between his nomination and that of G. Harrold Carswell shortly thereafter.

Carswell, then a Florida appellate judge, was-and still is-considered by many to have been unfit for the court. Following his own Senate rejection, he unsuccessfully sought his state's Republican Senate nomination in 1970.

"I'd just as sooon not be linked with Carswell," Haynsworth said. "I wasn't in Mr. Carswell's situation. He didn't even have the support of his own judges, and he made a foolish decision later to run for office."

Duke's VanAlstyne did a similar review of Carswell's cases and found "his work product was thoroughly mediocre. Nobody ever suggested that of Haynsworth."

Yet, when VanAlstyne made another appearance before the Senate Judiciary Committee to oppose Carswell, the atmosphere was "almost congratulatory" for the Florida judge in contrast to the hostile reception given Haynsworth a few months earlier.

Carswell initially received the warm welcome that should have been Haynsworth's, VanAlstyne said, laughting, "because they assumed the President wouldn't make two mistakes in a row." CAPTION: Picture, Judge Clement Haynsworth Jr., 10 years after Supreme Court rejection. UPI