Housing and Urban Development Secretary Patricia Roberts Harris agreed yesterday to permit the City of Alexandria to demolish 64 deteriorating Cameron Valley publis housing units in the city's western section, according to reliable sources.

Final details of the agreement are still to be worked out and the fate of 10 of the 74 Cameron Valley housing units Alexandria wants to tear down has not been decided. Nevertheless, the secretary's decision appears to have ended a bitter dispute between HUD and Alexandria over the city's right to raze a portion of the 328-unit World War II era project constructed for "temporary" use during the war.

Harris' decision to permit the demolition of units in the Cameron Valley project is in direct contrast to her actions three weeks ago. Then she ordered that the boarded-up Baber Village low-income housing project in Prince George's County must be renovated rather than torn down. That question now is in the U.S. courts, with Baber Village having become a symbol of Prince George's opposition to concentrated low-income housing.

In the Alexandria case, the city has argued that it made an agreement with the independent Alexandria Redevelopment and Housing Authority, which owns the project, to allow it to tear down 74 units in Cameron Valley in exchange for the construction elsewhere of 174 new units for the elderly plus 50 new units to replace those that would be lost because of Metro construction.

Alexandria has contended that the Cameron Valley units, which house about 200 residents have severe structural problems and frequently flood when it rains. (KEY OFF)(KEYWORD)s in the Baber Village case, HUD has contended earlier that Cameron Valley units could be rehabilitated and that Alexandria should doe this although the city had promised to relocate all the residents in public housing elsewhere in Alexandria.

At a private 35-minute meeting yesterday with Alexandria Mayor Frank E. Mann and Rep. Herbert Harris (D-Va.), however, Secretary Harris indicated that further inspection by HUD had revealed that 64 of the 74 units the city sought to demolish were not fit for human habitation and should be torn down, sources said.

Since last October, there has been a ban on demolition of HUD project without the personal approval of the secretary. "She looks at demolition as a very last resort," an aide to Harris said yesterday.

Alexandria officials refused to comment on the meeting with the secretary yesterday because some final issues, including the fate of the 10 units, still had to be resolved and the plan must be presented to the City Council. It was not clear that the council would take a formal vote on the matter.

The demolition issue upset Alexandria officials, who repeatedly told HUD that on a proportional basis the city has more publicly assisted housing than any other Washington area jurisdiction. In a June letter to Harris, Alexandria City Manager Douglas Harman quoted the 1974 report of the Northern Planning District Commission: "Alexandria has always led the way in suburban metropolitan Washington area in providing housing for low and moderate-income families."

Alexandria currently has 2,010 publicly-subsidized housing units, with an additional 51 units scheduled for completion later this year.

With the continuing expansion of the city's old and historic district into the surrounding low-income neighborhoods and the closing of the 2,100 unit-Shirley-Duke and Regina apartment complexes, low-income families are having a particularly hard time finding housing they can afford in the city.

Harris' decision yesterday reverses the ruling of Dec. 16, 1976, by HUD's Washington area office denying the city the right to demolition because in its view, the units could be rehabilitated.

Alexandria questioned HUD's legal right to prevent demolition because it said that the agreement between the city and its housing authority is legally binding. HUD said that it had an interest in the case because it gives the city an annual grant to maintain the project.

Yesterday's meeting with Secretary Harris was arranged by Rep. Harris, who said afterwards that with so many cities and mayors in this country, "I think it's great that the secretary could take the time to sit down and discuss the situation with Mayor Mann."