In what is being viewed as an ideological purge, the No. 2 man at the Department of Education has been forced out of his job. Undersecretary William C. Clohan Jr. abruptly announced his resignation yesterday, after reportedly incurring White House wrath for making a comment about tuition tax credits 10 days ago.

There was no official explanation for his departure, but sources said conservatives in and out of the administration felt he was "too liberal" and not a strong enough advocate of the administration's attempt to dismantle the newest Cabinet agency and cut federal spending in education programs.

Secretary of Education Terrel H. Bell asked for the resignation after returning from a White House meeting Wednesday, sources said, adding that the dismissal wasn't Bell's idea.

Clohan, 33, couldn't be reached immediately for comment. Bell told Associated Press that he couldn't speak for Clohan, but added: "Bill has served the Department of Education well over the past year since he arrived and we'll miss him." In response to a question, Bell said he has "no immediate plans to leave" the department.

Clohan's firing came the same day in which conservative leaders Richard Viguerie and Howard Phillips called for Bell's resignation "for failing to stop the flow of federal dollars to radical groups."

Viguerie and Phillips said in telephone interviews yesterday that they didn't know Clohan personally. Viguerie said he understood that "a lot of people felt he Clohan was a big part of the problem over there, and were unhappy that he ran the department." Phillips said he was more concerned about the administration's policies, and added, "Frankly, as they said in the French revolution, we want more than that."

The dismissal leaves as Bell's ranking subordinates general counsel Daniel Oliver, who used to work at the conservative National Review magazine, undersecretary Gary L. Jones, a known internal foe of Clohan's, and acting deputy undersecretary Charles L. Heatherly, formerly of the Heritage Foundation.

Education lobbyists expressed shock at Clohan's firing. "It's a shame. It looks like the ideologues have taken over the department," said Charles B. Saunders, chief lobbyist for the American Council on Education, a college umbrella group. "Bill Clohan is an honest, decent man. Whatever credibility the department had left it owed to him."

"It looks like the right wing is trying to clean out the department of anyone even somewhat of a moderate," said Allan S. Cohen, chairman of a coalition to save the department's Cabinet status.

August W. Steinhilber, associate executive director of the National School Boards Association, said, "I'm sorry to see Bill leave. He has been a person who understands the operation on the Hill and in the field. I wish he were still there."

Rep. John M. Ashbrook (R-Ohio), the ranking minority member of the Education Committee, said the firing was a surprise to him. "He's one of the most competent people I ever worked with. I'd be at a loss to find a reason why he should leave."

Criticism against both Bell and Clohan has been building for months, some administration officials said yesterday, because of their perceived failure to push the president's proposed budget cuts.

Bell, for instance, was quoted as telling the nation's chief state school officials last week that he has "been very careful not to be out front expressing Ted Bell's philosophy of education, which would not represent the point of view of this administration."

The last straw for Clohan may have been his comment to the Associated Press after that same gathering. He said President Reagan had approved the outlines of a tuition tax credit proposal. White House officials said later that the plan wasn't complete and presidential counselor Edwin Meese III reportedly called Bell to complain about Clohan's comments. The fiscal 1983 budget said such a proposal would be sent to Congress this year, but its passage was considered unlikely because it would cost the Treasury billions of dollars at a time of rising deficits.