The Reagan administration continued to whittle away at the Housing and Urban Development Department's budget for fiscal 1985. HUD's outlays were sliced from last year's $16.5 billion to $13.3 billion this year, with most cuts occurring in subsidized housing.

Gone is the expensive "Section 8" program for construction of subsidized housing. The administration wanted to replace it with 87,500 "vouchers" to subsidize the rents of poor tenants; Congress approved 37,500 vouchers.

The shift to vouchers, if expanded in future years, could produce big savings because vouchers would not last as long or be adjusted for inflation as often as current rent subsidies. Nonetheless, HUD officials can say they are serving more low-income tenants.

This year's figures on the halt in housing construction are in some cases more dramatic than they appear, because the outlays are made over 20 to 30 years. It would take a new administration several years to restore pre-Reagan levels.

The administration tried to cut one of the most popular programs, Section 202 grants for housing the elderly and handicapped, from 14,000 to 10,000 units. Congress did not go along.

Subsidies to operate 1.2 million public housing units were reduced about $500 million, through rescissions and cutbacks, to $1.1 billion. Modernization money for public housing stayed at $1.5 billion. Broader proposals, such as cutting off subsidies for up to 40,000 vacant units, are off until next year.

All told, the administration sought to cut subsidized housing, the third-largest federal welfare program, by about $2 billion more than Congress would accept.

The new budget also includes two small but visible compromises. The administration got $300 million for rental rehabilitation grants, a pet project of Secretary Samuel R. Pierce Jr. Congressional Democrats got $315 million for a small program of largely middle-income housing, with cities and developers putting up about half the funds. The money for both programs is to be spent over two years.

The big urban aid programs remain at existing levels, losing ground to inflation. These include $3.4 billion in community development block grants and $440 million in urban development action grants.