The arm of the General Services Administration in charge of buying federal supplies has been piling up a record of late that would make a businessman proud.

Back orders from its customer federal agencies have been reduced from 400,000 in October, 1981, to 43,000 this past January, while the time it takes to deliver a product has been cut from 12 to five days, on average. Suppliers are delivering orders late only 10 percent of the time, compared with 23 percent of the time previously. And the changes have been accomplished with a $13 million budget cut and 534 fewer employes.

Only three years ago, the Federal Supply Service was the agency's naughty stepchild, the focus of congressional investigations and Justice Department probes. As a result, 28 federal employes were convicted, a number of commercial suppliers were barred from selling to the government and 24 contractors were sentenced to jail.

But since then, the service has undergone an overhaul and a name change. A year ago, Administrator Gerald P. Carmen renamed it the Office of Personnel Property, but that proved too confusing so last month it was renamed the Federal Supply and Services.

"The Federal Supply Service had runaway costs, poor performance and one crisis after another," said Carmen. "We thought that . . . as a wholesaler, a properly run supply system can be cost-effective."

"Carmen came in here at a time that FSS was pleading for some kind of leadership," said Roger Daniero, its deputy commissioner. "He told us we would have to clean up the mess or he would recommend closing it down. We didn't take it as an idle threat."

After listening to career bureaucrats who had been stifled in the past, Carmen approved a series of changes designed to make the FSS operate more like a business. Now, Daniero said, "with renewed confidence from our customers, we're looking to more than doubling our one-third share of the $12 billion federal market" for tools, furnishings, paper products and other common supplies.

In most instances, the FSS prepares the contracts for purchases but does not buy the items. This year, it also is moving to use more general "commercial item descriptions" instead of writing its own detailed specifications for a product, and to use a new automated system to move supplies to customers more cheaply.

One major shift came in the service's Office of Contract Management last year when Director Lou Deprospero instituted a quality control system that he says "turns around an incredibly stupid way we had been doing business."

In the past, the GSA used to test a sample of a product it received: say, 35 of 1,000 widgets. If they passed tests at GSA laboratories, the entire shipment was accepted. If it turned out that half of the remaining widgets didn't work, it wasn't the contractor's responsibility.

Now, Deprospero said, the GSA has closed six of its seven testing laboratories, saving $3 million a year, and told contractors they will have to pay for their own quality control testing under government supervision. The remaining laboratory, in San Francisco, tests paints and abrasives.

As an example of the old problem, Deprospero pulled out a toolbox that the government had bought, and placed it on his couch. He unsnapped the two latches and the cover swung down and off, onto the floor. "We no longer contract for these," he said.