The Reagan administration has rejected Virginia's four-year-old college desegregation plan as a failure and ordered the state to come up with a new strategy, a move that some state officials said yesterday actually may give the state more leeway in meeting integration goals.

"I don't think they are going to let the state off the hook," said Cecil Carter, assistant Virginia secretary of education, "but I would interpret the move as offering some possibility of flexibility.

In a letter to Gov. Charles S. Robb, the office for civil rights in the U.S. Department of Education concluded that Virginia still has "several serious and longstanding problems" with the controversial integration plan adopted by former Gov. John N. Dalton in March 1978.

"The information available to us at this time indicates that the goals of the plan will not be achieved by the 1982-1983 academic year," said the letter, "It is imperative that the commonwealth indicate its intention to strengthen the existing plan."

The federal government has given Virginia two weeks to decide whether to accept its offer to renegotiate the desegregation plan. After that, the state would have 60 days to show how it plans to correct certain major deficiencies--such as the drop in the enrollment of black freshmen at the state's 37 colleges and universities during the last four years.

The Robb administration noted yesterday its commitment to "ending the vestiges of a dual system" of segregated colleges and agreed that Dalton's desegregation plan--long a hot political issue--has problems. "The administration believes that the office for civil rights has raised some important questions," said George Stoddart, Robb's press secretary.

Stoddart also suggested that the answer to the problem may be found in Robb's own recently stated educational policies. Those policies encourage blacks to begin preparing for college while in junior high school and call for improved programs to help them qualify for college work.

The letter from the office for civil rights, received in Richmond last Monday and released Thursday night, stressed that it will be up to Virginia to propose any alternatives to the current plan.

"We are ready and willing to work with you to develop additional or alternative measures to ensure that black citizens in the commonwealth are provided with equal educational opportunities and that the vestiges of the fomer de jure system are completely eliminated." the letter said.

Until school segregation was declared illegal in the 1950s, Virginia's colleges were divided along racial lines. Blacks were barred from 13 four-year colleges, including the prestigious College of William and Mary and the University of Virginia, while no whites were admitted to the state's two black institutions, Virginia State and Norfolk State colleges.

In 1969, the federal government warned Virginia and nine other states that their colleges were in violation of the Civil Rights Act. At first, Virginia resisted pressure to set specific integration goals and timetables. Eleven years later, faced with a possible cutoff of federal funds, the Dalton administration agreed to a plan that was supposed to bring Virginia into compliance by 1983.

The Dalton plan, acknowledged by its own drafters to have been hastily assembled, sets goals that would increase the number of blacks at traditionally white institutions, enhance the education offered by the traditionally black colleges and increase the hiring of black faculty members statewide.

By its own admission, the state has fallen short of its goals, especially in the last two years. For instance, by the fall of 1980, Virginia had hoped to enroll 1,650 black freshmen at the traditionally white institutions. Instead, the number was only 1,286 or 77.9 percent of the goal.

And while the total number of black students enrolled in the state's colleges has increased, blacks have not been entering college at the same rate as whites. Eliminating that disparity has been considered by federal officials to be a key criterion in judging the Virginia plan.

The office for civil rights has also continued to question whether Virginia has provided sufficient resources to Virginia State and Norfolk State, the predominantly black colleges.

These complaints and others were included in a motion recently filed in U.S. District Court in Washington by the NAACP Legal Defense Fund in the long-running lawsuit that many credit with prodding the federal government into enforcing college desegregation plans.

"Over the years, the federal departments have acted only when pressed," said Jane Fairfax, an attorney for the fund, "I would guess that is what happened here."

The NAACP has argued that the Reagan administration is to blame for the Department of Education's laxity in forcing Virginia and other states to comply with their own desegregation plans.

Barry M. Dorsey, associate director of Virginia's Council of Higher Education, said yesterday that he has noticed little change since Reagan took office. "It seems to be the same approach that has been going on since 1974," said Dorsey.