The housing recovery currently under way is so fragile that it's threatened by projected budget deficits and high unemployment, industry leaders warned yesterday.

Lenders and builders told a Senate Banking subcommittee that economic conditions that unnerve the financial markets could be fatal because even a slight rise in interest rates could bring the housing recovery crashing down. They said that an increase of only one percent in mortgage interest rates from the present level of about 13 percent would jeopardize continuing improvement in home sales.

The "housing recovery is threatened when the market gets jittery and drives rates back up," David O. Maxwell, chairman of the Federal National Mortgage Association--known as Fannie Mae--told a Senate Banking, Housing and Urban Affairs subcommittee headed by Sen. Jake Garn (R-Utah).

"Should mortgage rates average even one percentage point higher, or 14 percent, our housing forecast would show a peak rate of only 1.4 million" housing starts this year, Maxwell said.

Estimates of the number of housing starts that can be expected in 1983 range from 1.2 to 1.6 million.

Government and industry officials all agreed that a housing recovery really is under way. Federal Home Loan Bank Chairman Richard T. Pratt cited as proof a 43 percent increase in housing starts in December over the October 1981 low, a 54 percent increase in home sales in December over the September 1981 figure, and building permits over the million mark in five of the last six months.

Housing and Urban Development Secretary Samuel R. Pierce Jr. told the subcommittee the housing recovery already has meant that "construction workers are going back to work and more job opportunities are opening up." He said that, for every 100,000 housing starts, 142,000 jobs are created.

Three important conditions must be met, however, before the recovery can gain momentum, Maxwell said. In addition to interest rates below 13 percent, the federal deficit must be "substantially reduced and reliable financing from secondary market sources must be made available to the primary mortgage market," he said,

Pratt also reported that foreclosures and delinquencies in mortgage payments stand at about one-half of one percent of all mortgages, and added, "I wouldn't be surprised" by a continued rise in foreclosures to the historic high of one percent, a level reached during the 1960s.

Sen. John Heinz (R-Pa.), a member of the subcommittee, said research by the Library of Congress showed that foreclosures occurred on about 90,000 loans in the third quarter of 1980 and have risen in increasingly frequent jumps to a level of about 170,000 in the third quarter of last year.

Several witnesses voiced support for mortgage assistance programs that have been proposed to stave off foreclosures.

"Mortgage bankers are deeply concerned about the high level of mortgage delinquencies and foreclosures currently resulting from the sustained high unemployment levels," said James M. Wooten, president of the Mortgage Bankers Association and of the Dallas-based Lomas and Nettleton Co., the largest mortgage banking institution in the country.

"We would urge that the trigger . . . should be based on regional or local data, probably unemployment statistics, because delinquency and foreclosure problems have been concentrated in areas of higher than average unemployment," Wooten said.

An estimated $3.5 trillion will be needed to finance housing for the rest of the decade, said Leonard Shane, chairman of the U.S. League of Savings Institutions, and there is concern in the industry about where these funds will come from.