When Philip Abrams, nominated for the No. 2 job at the Housing and Urban Development Department, arrived at his Senate confirmation hearing a couple of weeks back he was greeted by 17 empty seats.

Alone at the center of the dais was Sen. Jake Garn (R-Utah), chairman of the Banking, Housing and Urban Affairs Committee, who served up a couple of friendly questions.

"The fact that there's no one here this morning indicates you'll have no problem with your confirmation," Garn told Abrams, who had been studying briefing books for two days. "Do not feel hurt--feel complimented."

Abrams' ascension to the undersecretary's job is part of an agency-wide shuffle that has reorganized some jobs out of existence. Abrams, an affable but aggressive former developer from Boston, was hired as deputy assistant secretary for housing and quickly moved up to federal housing commissioner. Some insiders are floating the name of Maurice Barksdale, deputy assistant secretary for multifamily housing, for Abrams' old job.

HUD also is creating two new high-level posts. A new position of assistant secretary to oversee more than 1.2 million public housing units will go to Warren Lindquist, who won high marks for cleaning up the mess in the now-defunct program to create "new communities." And leading HUD's image-polishing efforts is Robin Raborn, the acting assistant secretary for public affairs.

At a recent news conference Abrams said that his plans to revamp the Office of Housing would "reduce the amount of organizational layering" and get rid of some "anachronistic" programs. For example, he said, HUD doesn't need a construction standards office because most projects now defer to local building codes.

The moves also will cause 47 reductions-in-force. Abrams said this was unfortunate, but that HUD had "a continuing skill imbalance, where we don't have the right employes in the right slots."

But the American Federation of Government Employes is up in arms about the layoffs and the fact that it wasn't formally notified until the day of Abrams' news conference. "It's tantamount to a work speedup--the same amount of work being done by fewer people," one union official said.

On the local front, Terry Chisolm, director of HUD's Washington-area office, has been reassigned to a fair-housing unit in Philadelphia. Margaret White, HUD area manager in Richmond, is taking over the Connecticut Avenue office, which has been hard hit by layoffs and has enjoyed a less than stellar reputation for its record in overseeing troubled housing projects. * * *

SMOOTH RIDE . . . As part of a "Golden Fleece" award from Sen. William Proxmire (D-Wis.) last year, HUD Secretary Samuel R. Pierce Jr. won the dubious distinction of having the most expensive limousine of any Cabinet member.

Pierce was among 190 federal officials receiving chauffeur service to their homes at a cost of $3.4 million a year. Proxmire said the busiest car belonged to CIA Director William J. Casey, whose driver received $26,000 in overtime pay last year on top of a $20,000 salary. Pierce's Oldsmobile 98 diesel weighed in with the costliest lease at $8,088.

Proxmire, who authored a law barring many federal officials from using a government car for commuting, said that bureaucrats regard the cars as "a real status symbol. When we try to take the limousine away, they just buck like steers. I think they'd rather lose a billion-dollar program than a limousine."

A HUD spokesman said Pierce had wanted a cheaper General Motors car and settled for the more expensive lease after delivery problems with several area dealers. But when Ford came out with a discount program this year, he said, Pierce was able to get "a much swankier car," a Lincoln Mark 6, for just $3,075.