More than 55,000 builders and others connected with home construction left a convention of the National Association of Home Builders here this week with contrasting opinions about the state of their industry.

Pessimists tended to have a brighter view of their plight. The enthusiasm of optimists have been leavened by some of the downbeat predictions.

And their concerns relate directly to the ability of more Americans to buy new homes.

Jack Schwab, a Rockville builder, said he was buoyed by reports from economist Leonard Santow and other forecasters that the economy and its housing segment are stronger than seers expected them to be at this point. There was also near-unanimity that any recession will be far less severe than the 1974-75 housing holocaust.

The builders were told that mortgage rates might drop by as much as 2 percent before the end of the year -- or rise by several points. But there were few forecasts that mortgage rates would ever again fall below 11 percent.

The conventioneers were told that housing starts in 1980 will probably fall from the 1.79 million level of last year to between 1.35 million and 1.6 million.

But forecasters, builders and industry observers agreed that local housing markets now tend to vary greatly. House sales are strong in many parts of the South, Southwest and Far West and are depressed in the Northeast and Midwest. Yet there are exceptions in almost every area.

Lawrence Goodwin, a District condominium developer, said there had been no good news for him at the convention.

"It is becoming more and more difficult to provide housing for moderate-income families and virtually impossible to build rental housing without government subsidies," said Goodwin, who is president of the D.C. Builders Association.

Merrill Butler, a Southern California builder who was installed as president of the national homebuilder group, agreed.

"Costs of buildable land and materials continue to escalate," he said. "But even more importantly, virtually all homebuilders are also facing increased local, state and national restrictions, regulations and requirements that now account for an estimated 20 percent of the cost of almost every new house."

Butler said in an interview that builders need to be given more incentives -- in such areas as land use and density -- to build houses that can be afforded by young individuals and families whose incomes have failed to keep pace with mounting housing costs.

Jay Janis, chairman of the Federal Home Loan Bank Board, which has proposed that savings and loan associations be allowed to offer mortgages that can be renegotiated every three to five years, threw his support to builders on the issue of "excessive and costly goverment regulation."

Janis, a former builder and Housing and Urban Development undersecretary, said he is "seriously concerned" that measures must be taken to bring housing within the reach of the average buyer.

Area builders echoed those sentiments in interviews here. They maintained that buyers are slowly becoming accustomed to higher mortgage rates, rates that now often require as much as $600 a month to amortize a loan on a house cosing under $70,000. They also said that traffic at their model homes has perked up since Jan. 1.

William B. Regan said that his house sales and production in the Burke Centre area have been strong, despite the chill in the market from October to December.

The builders did not deny, however, that houses priced from $125,000 to $225,000 are selling slowly. But they did say that nearly all moderately priced condominium apartments and cluster houses priced under $70,000 are selling well, as are detached houses priced around $80,000 and big dwellings priced over $250,000.

On the other hand, sales have been sluggish in the heavily built town house market, especially in outlying suburban developments where houses are priced between $50,000 and $70,000.