The International Trade Commission lost the prime bargaining chip in its fight to refurbish its crumbling headquarters when the Senate voted to turn over the 144-year-old building to the Smithsonian Institution for use as an art museum.

The provision came as part of a last-minute amendment to the Public Buildings Authorization Act by Sen. John Danforth (R-Mo.) last Wednesday.

Senate Public Works Committee Chairman Robert T. Stafford (R-Vt.) pressed for the change, claiming that only 5 percent of the art works in the National Museum of American Art, the National Portrait Gallery and the Archives of American Art can be displayed at any one time because of a lack of space.

The Smithsonian had sought the building for a number of years. But when the Reagan administration came to power, building transfers between agencies was stopped unless the receiving agency could pay fair market value for the facility.

A top Public Works Committee aide said that an "inopportune" letter from GSA's new regional administrator, William F. Madison, prompted action by the Senate.

In a letter dated June 6, Madison told ITC Chairman Alfred C. Eckes Jr. that the agency's space requirements for the future had been "thoroughly reviewed by the real estate division" and could be satisfied by moving the agency to the nearby Bicentennial Building. The revival of the planned move--which Eckes opposed strongly--prompted ITC officials to mount a quick lobbying campaign among their congressional friends.

Although Madison wrote that the ITC's "concerns over the operating condition at the Bicentennial can and will be alleviated," Lorin C. Goodrich, the ITC's administrative officer, said "ventilation has been very poor, and security has been poor. Elevators are unreliable, and there have been times of infestation of roaches and rats . . . it is a very unclean type of place."

Susan C. Schwab, a Danforth aide, said the committee's position is that all federal occupancy in the Bicentennial Building should end when the lease expires in 1986.

Other problems with the Bicentennial Building, which now houses some ITC employes, include a "poorly designed underground garage and inadquate special-use space for hearing rooms for administrative law judges," Goodrich said.

"These problems, apparently, cannot be corrected," said Schwab.

"That fellow Madison ought to do better research before writing letters," offered a committee aide. "Saying 'here we come, ready or not' is not the way to go."

GSA has spent $1.25 million during the past three years to repair the Tariff building and planned to spend another $900,000 in the next three years. GSA also has been covering or removing potentially dangerous asbestos in the building. After the ITC is transferred, GSA will use Smithsonian funds to perform a complete renovation of the structure.