President Reagan yesterday denied using Education Secretary William J. Bennett to push out Supreme Court nominee Douglas H. Ginsburg and claimed that Ginsburg withdrew because of "harassment" and "clamor" from sources he did not identify.

"I stood by and declared that I would not withdraw him," Reagan told reporters. "He voluntarily made that decision on his own."

His account conflicts with authoritative descriptions of how Reagan's aides pressured Ginsburg to give up.

In addition, Sen. Orrin G. Hatch (R-Utah), one of Reagan's longtime conservative supporters, yesterday accused "gutless wonders" on the White House staff of pulling the rug from under Ginsburg. He did not name them.

Hatch said Reagan should find out who was undermining him and "bust 'em right out of the White House as soon as he can."

Meanwhile, Education Department officials reaffirmed their statements that Bennett did not misconstrue his conversation with the president Friday, after which Bennett called Ginsburg and urged him to withdraw.

"Every indication we had from the White House . . . was that they were grateful for what Bennett had done," said William Kristol, Bennett's chief of staff.

Amid recriminations over Ginsburg's withdrawal, White House spokesman Marlin Fitzwater said federal appeals court Judge Anthony M. Kennedy of Sacramento is the leading candidate for the court vacancy, although others will be considered.

Sen. Jesse Helms (R-N.C.), who had raised objections to nominating Kennedy, said yesterday that he is keeping an "open mind" and plans to meet the prospective nominee today. Kennedy also appeared to pick up support from conservative Sen. James A. McClure (R-Idaho), who said he could back Kennedy.

Sources said Attorney General Edwin Meese III and White House chief of staff Howard H. Baker Jr. agree on Kennedy, assuming that no one finds information that would disqualify him. White House counsel Arthur B. Culvahouse Jr. interviewed Kennedy yesterday as part of a renewed effort to scrutinze his background for possible controversies.

In remarks to reporters while touring United Way headquarters in Alexandria, Reagan tried to put distance between himself and Bennett's actions.

On Friday, Bennett's spokesman, Loye Miller, said Bennett had called Ginsburg and urged him to withdraw. He said that Bennett had told Reagan in advance of his intentions and that Reagan did not object. Another Education Department spokesman had quoted Reagan as telling Bennett, "Do what you think is right."

Fitzwater accused Miller yesterday of not telling the truth about the conversation. Miller "said things the president didn't say," according to Fitzwater. "I don't know what the exact words were, but I know he {Reagan} did not acquiesce and did not tell Bennett to do that.

"I know what the president didn't tell him," Fitzwater said, adding that he "absolutely" had talked with Reagan about it. Miller said the account of the conversation with the president came directly from Bennett, who "stands by it."

Reagan, who has often refused to acknowledge a personal role in such politically embarrassing controveries as resignations and failed nominations, told reporters that he took issue with "the way that story was told" about Ginsburg's withdrawal decision.

He also said Ginsburg used the "right word" in his statement, referring to "the clamor" for his departure.

Asked if he was embarrassed, Reagan said, "No, but I think there are others that should be." Ginsburg "chose to leave in view of the harassment that was coming," Reagan said. Asked who harassed Ginsburg, Reagan said, "I think it's very evident." Fitzwater would not say who Reagan meant.

Education Department officials stood by their original accounts. "No good deed goes unpunished," Kristol said. "I'm not saying that every press account was accurate, but Bennett did not misconstrue any of his conversations with the White House."

Fitzwater said he had told unnamed reporters late Friday and Saturday that the accounts were incorrect. Miller said Education Department officials had been careful to report that Reagan "neither encouraged nor discouraged" Bennett from placing the call urging Ginsburg to withdraw. Miller said Bennett would not have made the call if the president had discouraged him from doing so.

In the aftermath, Hatch was highly critical of the administration. "There wasn't a heck of a lot of support coming out of the White House for Judge Ginsburg," he said. "I wonder who's running the doggone place sometimes."

Also yesterday, Fitzwater denied a report in The Washington Post that Ginsburg went to the White House Saturday and expressed interest in remaining as the nominee but was pressured into withdrawal.

Sen. Strom Thurmond (R-S.C.), ranking minority member on the Senate Judiciary Committee, again urged White House officials yesterday to consider appeals court Judge William W. Wilkins of Greenville, S.C., passed over in earlier rounds.

Justice Department officials were examining other prospects, arguing that Kennedy's confirmation is not assured and that the administration could find a more appealing conservative candidate. Among those mentioned were appeals court judges Edith H. Jones and Ralph K. Winter Jr.

Staff writers Lou Cannon, Ruth Marcus and Barbara Vobejda contributed to this report.