Construction of new housing took its biggest drop in more than a year last month, slipping 14 percent from April's rate, the Commerce Department reported yesterday.

However, builders and housing economists predicted that the rapid decline of mortgage interest rates over the past six weeks will reverse the building slide, keeping sales and construction activity at healthy rates at least through the summer. Permits, an indicator of future activity, did, in fact, show an increase of 3 percent during May, according to Commerce.

"If this housing market stumbles and falls now, I'm going to be astounded," said James W. Christian, chief economist of the U.S. League of Savings Institutions.

The Commerce figures showed housing starts running at an annual rate of 1.66 million units in May, compared with 1.93 million in April. The rate for starts of single-family homes declined 10 percent, to 1.04 million, while that for buildings of five or more units plunged 22 percent. The annual rate for permits was 1.76 million.

A spokesman for the National Association of Home Builders said the group is not upset over the figures. "We didn't expect to sustain production" at the 1.9 million rate, he commented. NAHB President John Koelemij said that "builders are expecting a good summer for both sales and new construction."

And Warren Lasko, executive vice president of the Mortgage Bankers Association of America, said that, although he was surprised at the drop in starts of single-family homes, he views it as "a temporary aberration," noting that members of his association are reporting a "sharp increase in loan applications in the past few weeks."

Commerce Secretary Malcolm Baldrige, in a statement accompanying his department's figures, pointed to the improved interest picture. "Prospects for housing during the remainder of 1985 are favorable. . . . Single-family housing should benefit from the lowest commitment rate on fixed-rate first mortgages since the summer of 1980," he said.

The sag in housing construction starts during May appears to have resulted from relatively weak sales of new homes during April, several economists said.

"Builders had been starting single-family homes at a rate that was faster than the sales rate, and inventory was building up," said Timothy Howard, chief economist of the Federal National Mortgage Association. The "disappointing rate of sales in April caused builders to turn more cautious."

Howard noted that April sales followed a couple of months of rising mortgage rates. He said that mortgage rates were rising through the end of March and then leveled off during April at about 13 1/4 percent. Now they "have finally come down" so that sales "ought to be a good deal healthier, and if that happens, starts ought to turn up," Howard said.

Christian said that, according to his estimates, mortgage rates have come down a full percentage point since early May, "which is pretty extraordinary." He said it may be a little while before builders and the buying public realize the extent of the drop, but, when it does sink in, it should give the housing market a substantial boost.

"I do not anticipate seeing rates going back up anytime soon," Christian said. He said he expects the second quarter's gross national product figures to be "pretty grim," so "we've probably got through July and August and maybe a little longer with these blissfully mellow interest rates."

Howard said he believes rates "are close to the bottom," although there is room for mortgages for single-family homes to decline another half a point to the 11 1/2 to 11 3/4 percent range. But an upturn in the economy could cause rates to begin moving back up late in the year, he said.