A five-year national downswing in teen-age use of illegal drugs, as well as alcohol and tobacco, seems to have leveled off this year, while those who tried cocaine reached a record high number, according to a new federally funded study.

The annual nationwide survey of high school seniors, conducted for the 11th time by a University of Michigan team, found that the continuing decline in marijuana use seen since 1979 had stalled at levels almost identical to those seen in last year's study. Just over half reported that they had ever tried it, while one in four had done so in the previous month.

Declines in tranquilizer and barbiturate use had also slowed. And cocaine, tried at some point by 17 percent of the class of 1985 and by 7 percent in the past month, was up slightly in usage -- about 1 percent in both groups -- over last year, breaking a five-year pattern of stability.

"Clearly the lesson to be drawn from these findings is that we cannot take the improvement of recent years for granted," said Lloyd D. Johnston, one of the study's authors. "We are concerned that the steady, if gradual, progress . . . is showing signs of coming to an end."

He noted that the rates of illicit drug use among American young people today remain "higher than in any other industrialized nation in the world. Add to that the fact that the use of one of the most dependence-producing substances known to man -- cocaine -- is once again increasing and you have grounds for real concern."

The Michigan study surveyed 16,000 high school seniors in a representative sample of 132 public and private schools. Sponsored by the National Institute on Drug Abuse, it is a highly regarded barometer of teen-age drug use trends. Since 1980 it had shown, said Johnston, "an important turnaround" after nearly two decades of increases, reflecting greater interest in healthy life styles and concern about possible health hazards.

But in 1985, only three drugs showed continued decline -- amphetamines or "uppers"; methaqualone, a now-illegal sedative best known as Quaalude; and LSD.

While amphetamines are still the most widely used illicit drug after marijuana (tried at some time by 26 percent), those who had used them in the past month declined by 1.5 percent, to 6.8 percent.

Use of cocaine, now the third most frequently used student drug, is up slightly, regardless of sex, geography or college plans, said Johnston. This is particularly troublesome, he said, because usage expands with age. While only 10 percent of the class of 1976 had tried cocaine by their senior year, nearly 40 percent had tried it by age 27.

Overall, 6 in 10 seniors in 1985 admitted trying an illicit drug at some time, with 4 in 10 using an illicit drug other than marijuana. Active use in the month prior to the survey was reported by 30 percent (one-fourth lower than in the late 1970s, but no lower than in 1984).

Daily marijuana use -- at about 5 percent for two years running -- is about half of that in 1978. In 1985, daily alcohol use -- now at 5 percent -- increased slightly, while monthly and annual use dropped slightly. But 45 percent of boys and 28 percent of girls reported drinking heavily (five or more drinks in a row) in the previous two weeks. Daily cigarette smoking -- at a high of 29 percent in 1977 -- was at 20 percent in 1985, up slightly from last year.