IT'S HARD to imagine a whole warehouse full of evidence of scandal, but that's what the General Services Administration apparently has at its supply depot near Baltimore. As staff writer Ronald Kessier reported the other day, the huge GSA storehouse is piled high with metal office furniture to bought mainly fromArt Metal-USA Inc. and never used. Many of the desks, filing cabinets and such are defective or have been damaged by rain or bad handling. A lot has just been left to gather dust - while GSA has gone on buying more from the same firm.

No wonder GSA and Justice Department investigators are looking at the dealing with Art Metal, GSA's major supplier of metal furniture. And no wonder the government wants to review the notes, telephoned-calls logs and other records left at GSA by Robert T. Griffin, the agency veteran who was fired as deputy adminstrator in July. There is reason to believe that some of this material could ahed light on the Art Metal deals. After a few days of fuss, Mr. Griffin and the Justice Department reached an accord last Friday. Investigators will be able to scrutinize everything that might be relevant to their probes; Mr. Griffin will be able to retrieve materials that both parties agree are personal.

Mr. Griffin has not been named as a target of any probe. Indeed, the White House has ensconced him in a $50,000-per-year job in an effort to molify his chief patron, House Speaker Thomas P. O'Neill Jr. Given Mr. Griffin's long tenure at GSA, though, it was just a matter of time before the widening investigations would reach inot his domains. His willingness to cooperate is encouraging. Indeed, prolonged resistance could have had much the same effect as telling a child not to open a cookie jar. It could have made everyone more curious about what may be there.

The administration's relations with Art Metal have been somewhat bumper. GSA was recently reversed and rebuked by U.S. District Judge Harold H. Greene for trying to suspend its dealings with the Newark furniture firm precipitously. Despite all the questions that have been raised, GSA has not yet gone through the steps required to reach a formal verdict on Art Metal's fitness as a contractor. So GSA has to keep dealing with the firm - instead of cleaning out its warehouses.

That underlines the importance of conducting this whole inquiry strictly in accordance with the rules. General Services Administrator Jay Solomon and others are understandably anxious to root out and deal with the agency's problems quickly. President Carter's determination to clean up the agency is a further spur. But any shortcutting, however well-intentioned, could cause more embarrassing setbacks and possibly undermine the effort. Careful progress is shower, but safer, by far.