Education Secretary William J. Bennett, responding to weeklong attacks from education groups and bipartisan criticism from Capitol Hill, said yesterday that his support for student-aid cuts was well known early on by the senators who confirmed him for the job.

"I said flat out during the confirmation hearing that I recognized our proposal would have tough consequences for some people, but that I was in support of the proposal because it was targeted at the neediest," Bennett said in an interview with The Washington Post. "I was pretty candid on that."

Speaking in his fourth-floor office at the Education Department building, the secretary also made the following points:

* He has no intention of apologizing to students for saying some should consider selling their cars and stereos to help defer the cost of cuts in federal aid. "I did not demean students," he said, "I stated a fact, which is that for some students who are relatively well situated and who have been eligible for federal aid before, they will have to tighten their belts . . . "

* He is still waiting for education groups critical of the Reagan proposals to come up with ideas to reach the needed savings. He also rejected the idea of a student-aid spending freeze -- under discussion by some Republican senators -- saying it would not help save money in the program.

* While disagreeing with budget director David A. Stockman's choice of words, he generally agreed with Stockman's broader point that some colleges -- but not most -- concerned about financing their budgets may put "institutional survival" first, and "sometimes the concern for students takes a back seat."

* He feels that the nation's economic recovery put colleges and individual families in a better position to help students go to school. "It's helped the endowments of universities, and it's helped people's buying power," he said. "It's undeniable. That's the most important 'mega-fact,' if you will, about this whole debate."

In the half-hour interview, Bennett reacted most strongly to Sens. Robert T. Stafford (R-Vt.), chairman of the Senate education, arts and humanities subcommittee, and Sen. Edward M. Kennedy (Mass.), ranking Democrat on the Labor and Human Resources Committee, who both accused Bennett of misleading the Senate by not mentioning his support for student-aid cuts.

Bennett pointed to a sentence in the transcript of his confirmation hearing when, in answer to a question from Kennedy, he said, "I think if there are limited dollars, and my guess is under any proposal there will be limited dollars, those who are the poorest should be helped first."

Yesterday Bennett said, "I didn't side-step the general point with Sen. Kennedy or with Sen. Stafford, by saying if we had to choose I thought we had to choose help for the neediest first." He added, "That's the rationale of this proposal. That's the philosophical underpinning of it and there I was . . . pretty clearly on the record."

Bennett also said families hurt by a proposed $32,500 cutoff for federally backed loans still could qualify for a smaller separate loan program called "PLUS." That program is not widely used because -- unlike the general guaranteed student-loan plan -- recipients must pay the interest immediately on PLUS loans, not after the student finishes school, and the interest is higher.

The controversy Bennett generated this week continued yesterday when the president of the University of the Pacific in California withdrew an invitation for Bennett to speak at graduation ceremonies there, where Bennett was to receive an honorary degree. "We simply cannot honor a person holding these views . . . " wrote the president, Stanley E. McCaffrey.