Virginia Gov. John N. Dalton, under mounting pressure from conservatives over his agreement to end a prolonged college desegregation dispute, today backed away from his earlier support for the widely heralded settlement.

Dalton, who had made solution of the dispute with the Department of Health, Education and Welfare one of his top priorities, said he is now willing to help two Norfolk schools fight the issue in court.

His statement, at a State Capitol news conference, came as he was under increasing attack from college officials and his predecessor, Mills E. Godwin, for appearing to bow to HEW demands that Virginia eliminate vestiges of its once-segregated college system. Godwin, who fiercely resisted HEW, publicly attacked Dalton last weekend over his stance.

Dalton said he still hopes that boards of predominantly black Norfolk State University and predominantly white Old Dominion University can agree to meet HEW's request that the schools -- located about four miles apart -- eliminate nine duplicative courses.

However, Dalton added: "If the boards and HEW are not in agreement by Jan. 6, we are prepared to defend the schools in administrative hearings or in court if that becomes necessary."

Dalton's statement of willingness to fight HEW in court came after his negotiated agreement with the federal agency was criticized by Godwin and members of the state Council of Higher Education. The council, meeting in Fairfax County, passed a resolution this week calling on Dalton to inform it well in advance of proposals that might change instructional programs at state colleges.

Dalton said yesterday that he does not believe the proposed elimination of course duplications at the two Norfolk schools would affect the school's missions and said he had insisted that the council had been kept fully informed of his negotiations with HEW from the outset.

Virginia governors and HEW have been debating college desegregation plans for a decade. Godwin refused to accept racial timetables for integration proposed by the federal agency last year, saying that the timetables amounted to "racial quotas" for enrollments at state colleges.

Dalton supported the Godwin position when he ran for governor in 1977, but after taking office he told Attorney General J. Marshall Coleman and Secretary of Education J. Wade Gilley to try to reach an agreement with HEW.

The plan the state and federal officials approved sets targets for increasing black enrollments at predominantly white colleges and white enrollments at two predominantly black schools. Because of the governor's limited authority over Virginia colleges, many elements of the plan required approval of school governing boards. A scholarship plan intended to lure black students to predominantly white schools and white students to predominantly black schools will require approval of the General Assembly.

"It concerns me that those nine courses jeopardize the whole plan" Dalton said today "but I am pleased that we have narrowed the area of legal controversy."

HEW is under a federal court order requiring it to come up with new desegregation plans in Virginia and other states or cut off federal aid to colleges in states that refuse to approve acceptable plans.

Federal aid to higher education in Virginia exceeds $75 million a year. Dalton said he does not know whether HEW would attempt to cut off aid to the entire system if only the Norfolk aspect of the desegregation plan is rejected.