Alexandria's population is becoming older and wealthier, but faces continuing problems with a decreasing supply of rental housing, increasing crime and persistent drug use among some of its younger residents, according to the city's newly released annual report.

The 40-page document indicates that trends of the past few years are continuing. That means, among other things, that the public schools will continue to serve fewer students at what is expected to be a higher cost.

The report also indicates that rents will probably go up as older rental complexes are razed or converted to condominiums and as more affluent residents replace those of more limited means.

Specifically, the report states that about 104,500 people live in Alexandria, fewer than the 110,000 that city planners estimated last year.

The figures, from the U.S. Census Bureau, are being challenged by the city officials who contend the population decline is a temporary aberration caused by the closing of the Shirley Duke-Regina apartment complex in the city's West End.

That complex, now known as Foxchase, is expected to reopen this year with as many as 5,000 residents, city planners estimate.

Among city residents, the number of people 65 or older increased about 8 percent to 14,448. The estimate was based on Medicare enrollemnts, the report states.

However, federal employes can be exempt from Medicare, and since many current and retired federal workers live in Alexandria, city planners say the proportion of elderly people in the city may be much higher.

School enrollment declined 3 percent in 1980, to 11,099. Since 1973, enrollment has declined 25 percent. However, the report states that the racial balance within the school system has apparently stabilized, with 5,291 blacks, 4,786 whites and 1,022 children from other racial categories.

According to the report, nearly one out of every four Alexandrians earns at least $25,000 a year, with the percentage of residents earning less than $15,000 a year continuing to fall. These figures lie behind some planners' optimistic assessments of the city's financial future.

Adjusted gross income also continued to rise in Alexandria, to $24,691 for a couple and a median figure of $14,252 for all city residents. By contrast, the median adjusted gross income figure in Arlington was $14,821, and in Fairfax, $17,775.

Per capita personal income in Alexandria is $12,208, which is higher than the $10,259 areawide figure. The Alexandria figure is exceeded only by Arlington, where per capita income is $14,675, the report states.

The supply of rental units in the city fell 8 percent last year as 2,315 units were taken off the market. At the same time, the number of condominiums and attached single-family homes each rose by nearly 11 percent. The supply of detached, single-family homes fell by 3 percent. The average price of a residential unit in the city is now more than $90,000, officials said.

The implications of the trends in housing show that older and poorer residents might expect to be displaced from their aging homes and apartments as rental units are upgraded, converted to condos or demolished to make way for new construction.

Serious crime in the city increased 21 percent last year, with the greatest increases in burglary, larceny and auto thefts. Auto thefts showed the greatest increase, jumping 45 percent, from 491 in 1978 to 712 last year.

In the area of violent crimes, there were 12 murders last year, compared to 11 the year before, and 44 rapes in 1979, compared to 41 in 1978.

Visits to the city's substance-abuse programs, which are designed to eliminate or control drug abuse, are currently running 20 percent over capacity. Police officials say Alexandria is experiencing the same increase in drug use that other jurisdictions are seeing.