Just a few years back, they called it "Camp AU" -- The American University -- a sprawling 74-acre campus in Upper Northwest Washington where students could frolic through four years of higher education majoring in parties, beer and fun.

The drab, aging library was a ghost town. Stereos blasted disco music at all-night parties in dormitories and students periodically hurled beds out of windows.

But much of that has changed today. American University, as unobstrusive in its geographical setting as in its academic standing, is now striving to eradicate both the image and substance of its recent past and establish a solid reputation for demanding academic excellence and attracting bright students.

The new seriousness seems to be taking hold. "I find more people studying," says Marianne Malley, 21, a brown-haired senior from Spring Lake, N.J., who has been around long enough to see some of the changes. "You don't find as many people goofing off." Malley's sentiments are echoed by other upper classmen throughout the campus.

Yesterday, Richard Berendzen, a major architect of American's new look, became the university's 11th president in inaugural ceremonies at Washington Cathedral, less than a mile from the university's campus at Massachusetts and one Nebraska Avenues in Spring Valley, one of the city's wealthiest neighborhoods.

Berendzen, who has served in various administrative posts at AU since 1974, now takes over the 87-year-old Methodist-founded university with its bustling 12,500-member student body. But he also inherits one of the lowest paid facilities in the area and faces stiff competition from the other four major liberal arts institutions in the city -- Georgetown, George Washington, Howard and Catholic universities.

Berendzen vividly remembers what he found when he came to AU six years ago as a physics professor and dean of the College of Arts and Sciences, Practically everyone who applied was acepted. Final exams depended on the whim of professors. A schedule of final exams did not even exist.

Today, "The message I carry to students is that if you want a place to party and play, don't unpack, don't move in," said Berendzen in a 90-minute interview Wednesday. His boyish face and informally belie a no-nonsense administrator striving to put AU on the acadmeic map.

And students are getting the message. Sophomore Thomas L. Rosenberg of White Plains, N.Y., who sat strumming a guitar on the campus this week, said he had expected "a party school, a perpetual fraternity party," and instead found "if you wanted to party, you could. But if you wanted to study and work hard, you can."

"Five-page papers used to be common," said Conni Goodwill, editor of the campus newspaper. "Now it's 25 pages. They used to be reserved for special seminars."

In the past and even today, American University is the second choice for many of its students whose solid C averages and mediocre College Board test scores deny them acceptance to the Ivy League or other top-of-the-line colleges that they prefer. Other students are drawn to AU because of its reputation for good internships in government offices.

The atmosphere started changing about four years ago. When enrollment dipped in advance of a general decline forecast in the college-age population, many administrators at AU intepreted this as a rejection of its relatively high tuition costs and relatively low-quality education.

At the urging of Berendzen (then provost at AU) and others, the board of trustees decided it had to offer an education commensurate with the university's high costs, if it were to survive. (Tuition, room, board, books and incidental expenses now cost about $8,000 a year.)

Admission standards became tougher. Classloads increased 25 percent. Berendzen ordered an end to grade inflation. A new library went up. Quiet hours and a strict conduct code enforced by a student court calmed the dormatories. Last year, vandalism costs fell to $20,000 to $30,000 -- $80,000 less than the previous year.

The university became one of the first in the nation to require that all students pass a test proving a basic competency in reading, writing, and mathematics, including algebra, as a requirement for graduation.

"My concern was that students [often] graduate from here and from universities around the country with good grade point averages [but] are funtional illiterates," Berendzen said.

While the university puts its academic house in order, its old image remains with high school counselors at some leading schools because "reputations tend to linger," says Irene Pollin, a member of the AU board of trustees and wife of Washington Bullets owner, Abe Pollin.

"It's labeled as a college that's relatively easy to get into," said the college placement counselor at a leading New England private school. "Our children don't apply to American. They go to Georgtown if they want to come to Washington."

A counselor at a well-regarded Montgomery County high school agreed, but added that she had noticed a change in the last two to three years. "It's become more selective," she said.

"I don't have to use it any more for the kid who has a decent academic record who couldn't get in any college," said a couselor at a well-known New York area high school. "Now they have to have some kind of credentials." The counselor, like all the others interviewed, asked not to be indentified by name or school.

In the university world, reputations grow in part by the number of applicants who are rejected.

According to Berendzen, six years ago, 76 percent of all those who applied for admission to AU were admitted. Last year, the figure was down to 67 percent compared to an acceptance rate of 73 percent at George Washington University, 60 percent at Catholic University and 34 percent at Georgetown, according to The College Board, a nonprofit organization based in Princeton, N.J.

Two years ago, the average freshman at AU scored about 950 on the college board tests. Today, the scores of entering freshmen are up 65 points.

But AU is the only major area university lacking a Phi Beta Kappa chapter. The university was rejected when it applied six years ago. New chapters are voted on every three years.

AU's upgrading effort extends also to its faculty. However, because the average age of the 432 full-time faculty members is 42 and 80 percent of them are tenured, there is little room for personnel changes.

To prevent the faculty from becoming stagnant, Berendzen has launched a major faculty improvement program. The school recently received a $300,000 grant from the Mellon Foundation to improve teacher effectiveness in the College of Arts and Sciences, the university's largest academic division.

Also, the 42-member board of trustees collected $250,000 from its own pockets to create a distinguished scholar-in-residence program that will bring big name educators to the campus to teach for one year.This is a technique often used by universities as a drawing card for students, media attention and a stimulus for a faculty.

American's quest for quality means admitting fewer students at a time when the costs of running a private university are going up. This causes real problems because American is heavily dependent on tuition, receives little from its alumni and has only a $5 million endowment. "It's less than what many universities make on the interest of their endowment," said Berendzen.

Berendzen forecasts a deficit ranging from $6 million to $18 million by 1986.

Eighty percent of the school's $50 million budget comes from tuition. The faculty is one of the lowest paid in the Washington area, and Berendzen has promised to correct that.

"What AU needs desperately is a great infusion of funds," Berendzen said. "All the problems come down to that five-letter word -- money.

What the university also needs, many students and faculty say, is an infusion of campus spirit and unity, a difficult commodity at a school where there is no football team, where the basketball team plays at a gymnasium at Fort Myer in suburban Virginia and where half of the undergraduate students commute.

"It's a very separated school," said Janlyn Mattingly, 20, a junior from Kentucky. "There is no cohesiveness . . . As a result it's a very cliqueish school. You don't say you're proud of American University. You're proud of your department."

Berendzen and many students believe this lack of sense of belonging to the whole university is also a major reason for the dearth of alumni financial support.

To foster more communication among students, next month the campus government will begin mailing a monthly newsletter to each student, listing actions by the student government, future meetings and a schedule of campuswide events.

Berendzen wants to build a $5-$8 million combination field house and student union building as a gathering place on the campus, but there is no money yet.

Residents of Spring Valley, the wealthy enclave that surrounds most of the campus, are wary of such talk because the field house to them means rock concerts, more traffic congestion and more parking problems, according to Lois DeVecchio, chairman of the Spring Valley-Wesley Heights Advisory Neighborhood Comission.

Except for complaints from neighbors about students taking their parking spaces, the university gets along well with the surrounding community, according to neighborhood leaders.

The major reason for the good relations is that the university has rarely grown beyond its original borders unlike expanionist George Washington Univesity, which has embarked in recent years on a land acquisition program in neighboring Foggy Bottom.

Of American University's new president, Berendzen, one prominent local educator summed it up: "He's one of the most able academic leaders. He's the bright hope for American Universtiy at this point."