The federal government has withdrawn a $9.1 million grant to improve the property around the General Motors Corp. Broening Highway plant in Baltimore because the nation's largest automaker could not tell the government when it would go ahead with its $220 million expansion plans.

A spokesman for the department of Housing and Urban Development said the department was supposed to have received by last April a legally binding commitment from the city that the GM plant would be expanded, but delayed that deadline "to give the city and GM every opportunity to go forward."

Last August, however, GM announced that it was delaying the expansion and then in December told the city it did not know when or if the expansion would be made.

A General Motors spokesman said yesterday that the company decided to postpone expanding the plant for new front-wheel drive cars until market conditions improve, but has no immediate plans to leave Baltimore.

"This project has been put on hold," said William H. Noack, regional public relations manager for GM. "It has not been canceled." The company also planned a possible $220 million addition of a painting system, Noack said.

The funds withdrawal will have "no effect on the current state of affairs," in the city, a Baltimore city official said. But he said General Motors cannot expand without the improvements that would have been provided by the federal grant.

"We knew the Department of Housing and Urban Development was under pressure to put the funds into something else . . . . The HUD announcement was almost a technicality," said David Hash, a senior vice president of the Baltimore Economic Development Corp. "We knew the grant was in jeopardy."

"The funds can't be released until the developer makes a commitment," Hash explained. "GM never said in a legally binding way they were going ahead with the project."

The expansion was announced in the summer of 1980 and was a boost to the already thriving Baltimore redevelopment program. The federal funds would have paid for road improvements to support the GM plant's expansion. The plant now makes Chevrolet Monte Carlos and Malibus and would have been converted to build new, small, front-wheel drive cars, Noack said.

But because of the recession, the Baltimore expansion and other GM construction projects around the country have been postponed. So far, 3,600 of the plant's employes are on indefinite layoff and 2,400 still are working. Noack said he couldn't speculate on when the project would go forward.

"We're dealing with a protracted economic turndown," he said.

When HUD withdrew the funds, it told Baltimore that it would "welcome and look forward to" the city's application for future grants, and HUD encouraged the city to resubmit its proposal when GM makes up its mind. Resubmitting the proposal would take from 90 to 150 days.