Virginia officials, who this week filed a $73 million claim against the manufacturer of hazardous asbestos found in more than 240 state buildings, have failed to use millions of dollars earmarked for asbestos cleanup, according to a senior legislator.

"We've known there was a problem since 1979 and we're doing nothing to clear this up," charged state Sen. Wiley F. Mitchell Jr. (R-Alexandria). "What ought to be treated as a crisis is being treated as a routine maintenance problem."

His allegations that state officials have ignored the problem came a day after the state attorney general's massive claim against Johns-Manville Co., which manufactured much of the asbestos, was filed in a New York bankruptcy court.

Some state officials, including Gov. Charles S. Robb, have disputed Mitchell's statement.

"My primary conclusion is that Virginia, at both the state and local levels, is progressing at a creditable rate and much faster than any of our neighboring states," Robb told Mitchell in a recent letter. "The problem, like many problems, is one of priorities, money and state-local relations."

"I don't think there are very many things out there that ought to have a higher priority," Mitchell said in an interview. Mitchell, a lawyer, said he became aware of the dangers of asbestos while defending his employer, the Norfolk Southern Corp., a railroad company, against almost 40 lawsuits from past and present employes alleging health problems because of contact with asbestos.

Asbestos, which is regarded as a potentially cancer-causing substance, has been found in state-owned buildings ranging from college dormitories to mental hospitals to the General Assembly office building. It was used extensively as insulation and fireproofing in buildings prior to 1978 when the federal government banned its use in most construction projects.

Concern over the hazards of asbestos has grown in recent years as medical experts have learned more about its long-range effects. Asbestos manufacturers have been sued by more than 20,000 individuals and dozens of state and local governments nationwide, according to reports by the Council of State Governments. Maryland and the District are among the governments seeking damages because of asbestos in their public buildings.

Mitchell said Virginia has spent only about $1 million of the $4 million the legislature has allocated for asbestos removal from state-owned buildings in the last three years.

State officials said they are uncertain how much of the $2 million approved last year has been spent, but a Robb spokesman said the governor did not request funds for asbestos removal from the legislature this year because the state has not spent money approved last year.

The spokesman noted, however, that Robb has asked for about $687,000 this year to press claims against Johns-Manville and other asbestos manufacturers.

Mitchell also has launched an assault on the state's system of identifying asbestos in its buildings. "In some cases they're doing it with a janitor and a mop," he charged. Many more buildings may contain the hazardous material than state officials have identified, he said.

Mitchell has asked the legislature this year to spend more than $1 million to require the state to use health specialists to inspect all state buildings by Jan. 1, 1987.

Richard D. Justice, assistant director of the state Division of Engineering and Buildings, said each state agency has used its own employes to inspect buildings and then used industrial hygienists to test sites where the material was discovered.

The existence of exposed asbestos, which can crumble and release dangerous microscopic particles in the area, has created major problems at many state facilities. At Western State Hospital in Staunton, the state's largest mental hospital, potentially dangerous asbestos has been discovered in 11 buildings.

When the hospital recently renovated one building that contained large amounts of asbestos covering, workers had to "put clear plastic sheets around the entire building and clean out all the asbestos," said L.F. Harding, associate director of the hospital.

Officials at the University of Virginia in Charlottesville estimate that the cost of removing asbestos from the 19 locations where it has been found in classrooms and other buildings could reach $12 million.

State officials said potentially harmful asbestos also has been found in other state facilities, including historic Gunston Hall in Fairfax County, a food service building at Southampton Correctional Center, the president's house at Tidewater Community College in Suffolk and the Virginia Museum of Fine Arts in Richmond.

The state's most recent surveys also show that as many as 38 school districts have not corrected asbestos problems discovered in classrooms.

The Environmental Protection Agency has required all school districts to identify asbestos problems and notify parents if any dangerous asbestos is found. It does not require the school systems to remove the material. There are no regulations for identifying asbestos in other public buildings.

Asbestos problems frequently can be alleviated by either encapsulating the material or by removing it.