THE SALE OF PUBLIC housing is a controversial issue, but staff members at the liberal Institute for Policy Studies apparently didn't realize just how controversial when they invited the conservative Heritage Foundation to take part in a discussion of the subject.

Stuart Butler, director of domestic studies at the Heritage Foundation, accepted the invitation to speak at the seminar, held this week, as did Gordon Cavanaugh, a Washington attorney and Farmers Home Administration director in the Carter administration.

Then Butler withdrew, saying the foundation was "concerned" that it would be seen as "the opposite number of IPS. Neither are we as extreme . . . nor do we get involved in the more activist issues" the Institute for Policy Studies espouses. "We want to make crystal clear that we are not a conservative IPS. . . . We are solely involved in writing and publishing papers" as a "traditional" research organization.

But Chester Hartman, senior fellow at the Institute for Policy Studies, characterized the Heritage Foundation's "order" to Butler not to "set foot" in the IPS offices for the seminar as being "quite childish."

Cicero Wilson, director of neighborhood revitalization projects for the conservative American Enterprise Institute, replaced Butler.

GROWTH IN NET operating income from office buildings was almost flat in 1984, the Building Owners and Managers Association reports.

The report reflects a downward trend over the past few years, and it said that operating income is not likely to grow for U.S. private-sector buildings in 1985 and may decline.

"The potential for any significant increase in net operating income will be predicated upon construction activity slowing to market demand," said Ronald L. Simpson, president of BOMA International.

Vacancy rates run as high as 25 percent in some cities, Simpson said.

He said the cost and availability of money will be the most influential factor in determining whether construction activity slows.

In addition to the overbuilt situation facing building owners, the survey said costs for energy went up 8 cents a square foot from 1983 to 1984 and property taxes increased 11 cents a square foot.

The statistics were gathered by BOMA from owners of 3,000 large office buildings covering 560 million square feet.

MONTGOMERY COUNTY'S planning board designated four sites in Silver Spring as historic last week but remained lukewarm about a half-dozen other candidates, including a ginger ale bottling plant, some garden apartments and a former car dealership.

The historic designations are part of an ongoing process to preserve most of downtown Silver Spring from high-rise development, said John Hoover, planning spokesman. Last year, planners created a historic district emanating from the corner of Georgia Avenue and Colesville Road in an effort to keep what they said is the "human scale" of one- and two-story shops and restaurants.

Now the board is selecting specific properties to include in the county's master plan as historic, a move that will prevent the sites from being altered or torn down. But planners and historians are in conflict over what properties should be designated "historic," because many of the properties in question are less than 50 years old.

"There is quite a difference of opinion," said Hoover, who noted that the board split its vote on many of the properties up for consideration last week. "The overall impression was that they were lukewarm about" the properties.

The board designated as historical the Jesup Blair House, a Greek revival home built in 1850 on Jesup Blair Drive; the Silver Spring/Acorn park, from which the city derives its name; Armory Place on Wayne Avenue, dating from the early 20th century; and the Old Silver Spring Post Office on Georgia Avenue, built during the Depression.

The board voted to leave another five sites off the master plan but to keep them on a "historical atlas" from which the board derives candidates for the master plan designation. These included the Canada Dry bottling plant on East-West Highway, the Spring Garden Apartments on Blair Mill Road and the Wolfe Building on Georgia Avenue, originally built as a car dealership. Two other properties, the Old Masonic Temple and the Little Tavern, both on Georgia Avenue, were dropped from the atlas.

PRINCE GEORGE'S COUNTY planners said last week that they will hold public hearings on a controversial proposal to triple the size of a school for drivers of tractor trailer trucks in a rural residential neighborhood next door to the site chosen for Konterra, an ambitious planned residential community.

Owner Charles R. Longo wants to add 16,266 square feet to a 9,600-square-foot building, and construct several more truck bays to accommodate 150 students per session on the school's 10-acre site off Contee Road near Laurel. The application also includes a request for permission to hold classes for 10 hours a day on Saturdays and Sundays.

The county planning staff has recommended approval of the plan on the condition that Sunday classes be dropped, because it said that the adjacent residents "should be afforded at least one day a week the peace and tranquility most residents in a rural setting such as this can enjoy every day."

Staff members reported to the county planning board that Prince George's policy is to encourage development that creates a skilled labor force and said the school has an 85 percent placement rate. The only other truck-driving schools in the area are in St. Mary's County and Newark, Del., they said.

But neighbors said they oppose the addition because the school already creates noise and is unsightly. The staff report on the application concurs that five acres of forest behind the school were leveled recently despite site-plan restrictions, and the area now is used to store trucks. The report also stated that the school uses a cul de sac in front of its main building to train drivers rather than a paved "training area" nearby.