Virginia education officials expressed pessimism today about getting the state's latest college desegregation plan off the ground this year after an expected agreement with the Office of Civil Rights collapsed.

State Education Secretary John Casteen told a legislative committee this afternoon that he would not seek any money for the first year of the proposed plan until a final agreement had been reached. Casteen did not know when negotiations over the plan would resume. The state proposes to increase black enrollment at the state's 13 predominantly white four-year colleges by 1,500 and improve the quality of education at Virginia's traditionally black state universities.

Appearing before the House Appropriations Committee, Casteen said, "The federal government had agreed in principle to the proposed amendments but that several details remain to be worked out" on the plan submitted by Virginia last September.

Reaction from another high-ranking state education official was gloomier. "I am pessimistic. There are always a few small details to be worked out," said Gordon Davies, director of the Council of Higher Education. "What was it Yogi Berra said? 'It ain't over until it's over.' "

Davies and Casteen had hoped to have an agreement in hand today so they could ask the General Assembly to fund part of the plan this year.

In draft documents, Casteen has estimated the first year costs at $1.8 million but that figure could change depending on the final agreement.

Virginia's long-running problem with college desegregation became an issue for the administration of Gov. Charles S. Robb last year after the Office of Civil Rights informed the state that an early plan, written during the previous administration, had been a failure.

The Robb plan, expected to cost $25 million over five years, was submitted to the Office of Civil Rights at the U.S. Department of Education last September. Negotiations with the federal government did not start until last month.

Casteen would not discuss today what problems federal officials had found with the plan, saying only that they were "quite small." But, like Davies, Casteen is eager for an agreement in time to submit a funding request to the General Assembly.

Any agreement coming after the assembly's Feb. 26 adjournment date would have to be funded either at a special session or during the 1984 session.

Technically, the deadline for budget amendments is 10 days away but Casteen was concerned today that the chance of finding additional funds during a year of harsh economies will grow increasingly slim.

"I was here to ask for the money today," he said.

"Any loose dollars that were around today were liable to be gone tomorrow."

The plan calls for further enhancement of programs and facilities at Norfolk State University and Virginia State University, two traditionally black public colleges.

The Robb plan came after the federal government declared last summer that Virginia's 1978 college desegregation plan was a failure.

By 1981, only two of the state's traditionally white colleges had met the goals of the earlier plan.

The new plan gives the state another five years to comply with a 1969 federal desegregation mandate.

The plan called for $1,000-a-year grants to blacks transferring from community colleges to finish their junior and senior years at the state's predominantly white four-year colleges.

Similar grants would be available to whites transferring to the historically black colleges.

The grant program would cost an estimated $2.9 million during three years.

Other elements of the plan include:

A $100,000-a-year program to attract white professors to Virginia State and Norfolk State; a $500,000 program for creating a pool of 10 "nationally distinquished" scholars who would agree to teach at one of the black schools, and a wide-ranging series of "enhancement" actions at black schools, including the construction of new buildings and higher administrative salaries.