The Maryland Court of Special Appeals ruled this week that portions of the top, sides and foundation of an oversized office building in Silver Spring will have to be removed to bring the building into line with county zoning codes, alterations that could cost the building's owner between $250,000 and $500,000.

Joseph Blocher, the attorney for owner Permanent Financial Corp., said that his client is planning to appeal the decision to Maryland's highest court, but that they also are working closely with county officials to reach a compromise to end the litigation.

"They Montgomery County building officials haven't given us a clear reading yet on what we would have to do, but if it didn't break our back my client might agree to do the work on it," said Blocher. He said that if they could reach an agreement, Permanent Financial would drop the court case.

Silver Spring residents first complained about the size of the $3 million building being constructed at the corner of Wayne and Cedar avenues more than two years ago, and work was later stopped by the county Department of Environmental Protection after it found that the building was 18 feet taller than it should be and exceeded allowable density by 26 percent and allowable setbacks by two feet.

Permanent Financial appealed to the county Board of Appeals, claiming that the county DEP had approved the building plans and inspected the building during construction, and that officials said nothing about any height or density violations then. Permanent Financial said it needed the variances because altering the building after-the-fact would impose undue financial hardship.

The Board of Appeals, however, ruled in late 1983 that Permanent Financial should not be granted after-the-fact zoning variances for the oversized building because the company has "misrepresented" the size of the building on the plans approved by DEP.

Last spring, Montgomery County Circuit Court Judge Stanley B. Frosh upheld that decision, saying that "the lot did not pose exceptional circumstances justifying the granting of variances and that Permanent Financial's hardship was created by its own actions."

The Maryland Court of Special Appeals, in a decision filed this week, agreed with Judge Frosh, upholding his entire decision.

Blocher said, however, that he believes his client still has special rights in the case that should be pursued to the Maryland Court of Appeals, the state's highest court.

"The fundamental issue in this case is whether the county bears any responsibility in what happened," said Blocher. "The lower courts have all said the county doesn't have any responsibility, and we say that isn't right. Other courts in other states say local officials have a responsibility in a case like this and maybe its time for Maryland to change." Blocher said that he expected his client might reach an agreement with the county within the next month.