The population of the District of Columbia continued to fall last year, dropping to 656,000, about 100,000 less than in 1970, according to new U.S. Census Bureau estimates.

The census bureau said the number of city residents last July 1 was down about 15,000 from a year earlier, or 2.2 percent, and had dropped by 13.4 percent since the 1970 census.

No detailed information on age, income, race, or family size will be available until after the 1980 census is conducted on April 1.

But both federal and city officials suggested that the downturn reflects a sharp drop in average household size as many families with children move to the suburbs and the number of births declines.

"The District isn't unhealthy by any means," said Donald E. Starsinic, chief of the state and national estimates branch of the Census Bureau. "There is still a lot of building. But the new units are expensive and there's a lot of renovation taking place . . . Middle-class people without children are coming in but there's a substantial out-migration of families."

In Virginia, the population last July 1 was 5,197,000, according to the new census estimates, up about 20,000 from mid-1978.

Maryland's population was virtually unchanged at 4,149,000 up just 1,000 from a year earlier.

Since 1970, Virginia's population has grown by 11.7 percent. This is more than double the growth rate for Maryland, which grew by just 5.7 percent from 1970 to 1979.

All of the new figures are estimates. They are based on several complicated formulas using birth and death statistics, school enrollment data and information from federal income-tax returns, Medicare reports and immigration service reports.

Starsinic said Washington's population loss is comparable to that for most other big cities. The D.C. figures are a year more up-to-date, however, because they are issued by the Census Bureau along with the estimates for the 50 states.

According to the new report, Washington now has a smaller population than all but four states -- Alaska, Wyoming, Vermont, and Delaware.

In 1970, there were 10 states that were smaller than the District. Four of them moved ahead of D.C. early in the decade -- Idaho, Montana, New Hampshire, and South Dakota. Last year North Dakota and Nevada also moved ahead.

Nevada's population reached 702,000 last year, the Census Bureau said, an increase of 43.6 percent since 1970, which is the fastest growth rate for any state in the union.

Overall, the population of the United States increased by 1,871,000 last year to 220.1 million, but the growth rate continued to be far less than it had been during the 1960s.

The fastest growth last year and throughout the decade was in the Rocky Mountain states and the South.

Seven states in the Northeast and the industrial Midwest lost population last year, including a drop of 97,000 in New York. Since 1970, New York's population has fallen by 592,000 or 3.2 percent.

Slight decrases also were reported last year in Delaware, Iowa, South Dakota and Alaska.

Marcia Kunen, a population analyst in the D.C. government's planning division, noted that despite Washington's substantial population loss, the number of households in the city has actually increased slightly since 1970.

About two-thirds of the city's househlds, she said, now have only one or two persons.

"There's a very different profile of households here than there was 10 years ago," Kunen said. "Very basic changes are taking place in social patterns and family formation . . . It's the household that's your basic unit of earning and spending. There's less population pressures now (in the city), but the number of households is stable or growing."