The New Right escalated its battle against the Supreme Court nomination of Sandra D. O'Connor yesterday with charges that either the Justice Department or O'Connor herself had tried to "cover up" her position on abortion. In the process, however, the conservative group angered another one of its traditional Senate allies, Sen. Gordon J. Humphrey (R-N.H.).

Humphrey said he had taken no position on the nomination but objected to the "hip-shooting of O'Connor's opponents. . . . The shouldn't categorize or stereotype someone without waiting for full hearings. I don't think they've done themselves any favors."

Sen Jesse Helms (R-N.C.) said yesteday that O'Connor would come here next week for a meeting with members of the Judiciary Committee, an unusual step to allow questioning by senators about her views on abortion.

Meanwhile, sources said the controversy was beginning to trouble the White House, which began an effort to prepare O'Connor for the confirmation fight ahead. The White House assigned its highly regarded chief lobbyist, Max L. Friedersdorf, to the case.

Day three of the battle began with a dispute about an internal Justice Department memorandum distributed by Conservative Caucus Chairman Howard Phillips and other anti-abortion activists at a morning news conference.

The memo describes O'Connor responses on the day before her nomination to questions on her record as an Arizona state senator. Kenneth W. Starr, a top Justice Department official, reported that O'Connor indicated "that she had never been a leader or outspoken advocate on behalf of either pro-life or abortion-rights organizations." But Starr also reported that she "has no recollection" of how she voted on a 1970 Arizona bill to decriminalize abortion.

Kathleen Teague, one of the anti-abortion speakers at the news conference, said the memo showed a "cover-up" by the Justice Department or O'Connor because, among other things, O'Connor co-sponsored the bill in question and voted for it.

That was when events began backfiring on the anti-abortionists. Legislative records in Arizona contradicted Teague's claim that O'Connor co-sponsored the bill, according to the keeper of the records, Greg Jernigan. cConsulting the records, Jernigan said the only abortion bill O'Connor sponsored was one giving doctors and hospitals the right to refuse abortions.

Newspaper accounts, however, confirmed Teague's statement that O'Connor voted for the decriminalization proposal in committee.

The anti-abortion group held its news conference i a room reserved in the Capitol by one of Humphrey's Senate staff members. That caused the second flap for O'Connor's opponents.

Humphrey, generally a staunch New Right supporter, had not been informed of the purposes of the news conference. When he found out, a press aide began calling reporters to disassociate the senator from the whole enterprise. Humphrey later said he was "quite upset" by the incident.

Humphrey said he has taken no position on the nomination and did not want to be identified with one. Beyond that, he said he objected to the "hip-shooting" of the opponents. "They're objecting to most of her votes as a senator," he said. "I know full well that votes can be misconstrued."

Humphrey's reaction illustrated what appears to be a delicate but clear shift in alliances on the right with implications beyond the O'Connor dispute. Many conservative politicians, including President Reagan, seem anxious to use this opportunity to publicly separate themselves from the far-right organizations that helped elect them and to identify themselves with a more moderate conservatism.

At this point, however, Reagan and the anti-abortion forces are avoiding direct confrontations. Phillips and his allies at yesterday's news conference took pains to place the blame for the nomination on aides to Reagan, who they say misinformed the president.

White House officials have, in turn, decided not to respond to the criticism themselves. Instead, that task has been left to the Justice Department and Starr.

Yesterday, Starr dismissed the "cover-up" allegation. He said the memorandum he wrote "accurately memorialized my conversations" with O'Connor. m

The memorandum does suggest that the administration might have been caught off guard by the abortion controversy. The telephone call to discuss her voting record came after officials had already spent hours talking to O'Connor and after Reagan had made the decision to nominate her.

Starr's inquiry appeared to be a last-minute response to preannouncement indications of trouble on the abortion question.

The next round of the fight is expected to revolve around the timing of the confirmation hearings. The anti-abortionists urged the Judiciary Committee to postpone any hearings until late September. The Reagan administration is pushing for hearings by late July.