A bitter controversy in Richmond over construction of a downtown convention center -- which roughly parallels a similar conflict in Washington -- led to cancellation yesterday of the Virginia city's plans to sell $32.2 million of bonds on Wall Street.

Following a court suit filed on Tuesday by opponents of the center citing alleged procedural problems, Richmond city lawyers and financial advisers in New York advised against going ahead with the planned offering.

Mayor Henry L. Marsh II denounced the lawsuit as an "irresponsible use of the judicial process to frustrate the city in the proper management of its financial affairs" and said the delay in selling bonds may cost city taxpayers between $4 million and $6 million.

Richmond initially had planned to sell the bonds last year, with $20 million of proceeds earmarked for needed expansion of sewer, water and recreation facilities, and $12.2 million for initial financing of the Project One convention complex, with adjacent offices and a hotel on several blocks of small buildings and parking lots near the Richmond Coliseum downtown.

A referendum drive by project opponents and subsequent state and federal court sutis delayed the bond sale. After appeals were exhausted and the city was upheld, plans were made late last year for yesterday's sale -- although the bonds were to be dated Jan. 15, 1978, in an effort to aviold another legal challenge by opponents.

The new court action this week apparently has blocked any attempt to sell the 1978 bonds, however. "We have to go back to Square One," said the city's finance director, Jack Lissenden, in an interview yesterday.

Lissenden said the bond sale cancellation also makes the legal case moot, although city lawyers were "confident" it would have been dismissed. The problem was that the city did not have enough time to seek dismissal and fight a likely appeal to a higher court, he said. The sale was scheduled for noon yesterday.

According to the Richmond finance officer, many of the capital improvements covered by the bond offering are in progress, and the city will need additional funds before the end of the year. Lissenden added that a new bond sale could be prepared within a month or two and that "we have enough cash to carry us... with real estate taxes due in June, we'll have $90 million by then."

The suit was filed by Charles Harper, head of the Richmond Independent Taxpayers Associations, who also had challenged the convention center project last year.

Mayor Marsh said that "due to the sensitivity of the bond market, the merest hint of controversy" could lead to higher interest costs or no bids from bond firms that sell such issues to the investing public. He said that in addition to higher taxpayer costs, yesterday's action could delay completion of some needed sewer and water projects.

Plans for a $100 million D.C. convention center near Mt. Vernon Square have been opposed by city residents and neighborhood groups who have sought a referendum on the issue here. Although the city plans no bond sale, the proposed expenditure of D.C. funds this year for initial work has been challenged in a Superior Court suit seeking to force a referendum. A majority of the D.C. City Council and new Mayor Marion Barry Jr. support construction of the Washington complex.

In Richmond, former city manager William Leidinger was fired last year in the wake of a conflict with the black majority of the city council that focused on concern that the city government was emphasizing commercial redevelopment at the expense of social programs.