The Carter administration, noting that housing has been one of the industries hardest hit by high interest rates, endorsed a limited program for both homebuyers and builders yesterday.

The National Association of Home Builders immediately expressed disappointment in the government action. Merrill Butler, NAHB president, said "the actions announced today do not alleviate the problems confronting the vast majority of America's homebuyers and builders."

Under the program announced yesterday, a family of four with an income of up to $27,000 would, depending on where they live, get a mortgage of up to $60,000 at an effective interest rate of 8 percent the first year. The rate would rise gradually to 11 percent.

The mortgage program represents a considerable expansion of the Sect. 235 home-ownership-assistance program which previously limited income eligibility to $16,000.

By making more people eligible for FHA-insured loans, federal housing officials hope to reduce the backlog of 340,000 unsold new homes. They also hope to spur construction of as many as 100,000 more new houses.

To assist builders, who are stuck paying up to 22 percent interest on unsold completed homes, the administration is proposing that they be allowed to convert a portion of their inventory to permanent financing at 14 percent. The maximum mortgage a builder could obtain through the FHA would be $67,500. The administration estimates approximately 135,000 of the 340,000 unsold homes can be refinanced this way.

Butler told a news conference the program is like "scoring a touchdown in a football game when your team is already trailing 50 to 3."

He predicted 1980 housing starts will fall below 1 million -- more than 160,000 below the starts in 1975, the worst year for new housing in the 1970s. Butler predicted long-term mortgage rates would decrease only 1 or 1 1/2 percent by the end of the year.

Although aid to the housing industry has been under consideration for several weeks, the administration until this week had doggedly opposed giving any signal to the credit markets that it was loosening its controls. But the easing of interest rates recently and congressional insistence on voting some aid to the construction industry apparently persuaded the administration to change its position.

The announcement of support came one day after the Commerce Department announced the second-largest monthly drop in housing starts, 26 percent, since it began keeping records. The U.S. League of Savings Associations, meanwhile, announced a $940 million loss for S&Ls, the first March loss in history. Total savings increases at S&Ls for the first quarter of 1980 were 88 percent below those for the same period in 1979.

The relief program, composed of both administrative and congressional actions, would require no additional appropriations for 1980 and a $75 million appropriation for 1981. The $160 million to be expanded is left over from a freeze put on funds appropriated in 1974-75 for the Sect. 235 home ownership assistance program.

The Senate plans to vote on expansion of Sec. 235 next week. If approved, the program could get started by late summer. Regulations for the builders' refinancing program are expected to be announced next week by HUD.

High officials made clear yesterday the administration never would have supported the so-called Brooke-Cranston program demanded by the building industry. This program, which requires the federal government to buy entire mortgages and sell them at a discount, would have meant a budget-busting outlay of $5 billion for each 100,000 units. Sect. 235 is aimed at accomplishing roughly the same objective of subsidizing rates for about $235 million up front.