Warren Richardson, whose nomination as assistant secretary of Health and Human Services had been challenged because of allegations of anti-Semitism, asked last night that the nomination be rescinded.

Richardson, denying again that his four years as general counsel of Liberty Lobby had made him unsuited for the HHS post, sent a letter to HHS Secretary Richard S. Schweiker declaring that "political realism" was forcing him to withdraw his name from consideration.

Richardson said that, while he had done everything possible to clear his name of charges of anti-Semitism, he "cannot allow a protracted political battle" to interfere with the work of the new administration.

Schweiker, in a letter to Richardson made public late last night, said he accepted Richardson's withdrawal "with regret" and declared that, "after careful review," there was "no convincing evidence" that Richardson had ever been anti-Jewish or racist.

The dispute over Richardson's nomination surfaced last week when Rep. Samuel Gejdenson (D-Conn.), the Anti-Defamation League of B'nai B'rith and the American Jewish Committee charged that Liberty Lobby was an anti-Semitic and racist organization.

They said Richardson's tenure as general counsel from 1969 to 1973, during which time Liberty Lobby's publications charged that Jews were financing communism and were involved in a Zionist plot to rule the world, should disqualify Richardson from the HHS job.

Richardson, in a memorandum to Schweiker aide David Newhall following his nomination, said that he realized soon after joining Liberty Lobby that it was "anti-Jewish and racist" but that he did not quit immediately because he needed the job.

"In retrospect, it became clear to me long ago that it was wrong not to have quit earlier," he said in the memorandum.

"I apologize for my inaction to all those who have felt the vicious racist and ethnic stings of the Liberty Lobby. I never participated in those Liberty Lobby activities. I never agreed with them. I found them then, as I do now, to be vile," the memorandum said.

Richardson's nomination as assistant secretary for legislation had been under review by Newhall for a week, and Schweiker had indicated he would decide early next week whether to press forward with it.

Had Richardson not asked that the nomination be rescinded, it would have faced a serious fight in the Senate where at least one prominent Democrat was prepared for an all-out battle to defeat it.

In his memorandum to Newhall, which became available Thursday, Richardson said he took the Liberty Lobby job during a difficult period when his wife had been injured in an auto accident that required five costly operations.

He said in the memorandum that, while he did not know at first of Liberty Lobby's "racist or anti-Jewish views," he soon became aware of them although he did not leave immediately because of the pressure of his family obligations.

Richardson contended in the memorandum to Newhall that, except for lobbying on Capitol Hill, he played little role in the basic operation of Liberty Lobby and that his title as general counsel was largely pro forma.

However, Warren Keller, a former Liberty Lobby official who worked there during Richardson's tenure, disputed this in a telephone interview yesterday.

Keller said Richardson appeared to him to have considerable authority -- not as much as the organization's founder, Willis Carto, but perhaps as much as board chairman Curtis Dall.

Moreover, Keller said, Richardson "always sat in on editorial conferences and [sometimes] made suggestions" on what should go into publications. "I don't recall his saying anything that he didn't want this or that in."

Many of the publications at that time contained accusations that Jews were financing world communism or that Zionism wanted to take over the world.

Keller said he never heard Richardson take issue with these positions, although sometimes Richardson "softpedaled" some of Carto's statements on race.

Two other former Liberty Lobby employes, who asked that their names not be used, said they recall that Richardson's general power and role in the organization was larger than described in published reports of his memo to HHS.

The employes said they recall that Richardson had been de facto head of the organization at times when Carto was away.