The freshman who once caught the only Harvard touchdown pass in the Crimson's traditional battle against the Yalies a quarter century ago quietly sipped Molson Golden Ale last night and sang along with his illustratious classmates and the Boston Pops:

"We'll fight for dear old Harvard 'til the last white line is crossed."

But this alumnus of the class of '54 was clearly different.

Sen. Edward M. Kennedy (D-Mass.) arrived an hour late in a chauffeur-driven limousine. His former classmates were shuttled in on school buses from Harvard's athletic complex.

As the Pops built to a crescendo on Grieg's "Piano Concerto in A Minor," he tried to sneak into the concert hall - his steps dogged by photograhers and reporters.

the whispers spread from table to table: "It's Ted."

The class of '54's most sterling alumnus had returned.

"Nice to see you,' he said, signing autographs, smiling at the wives. "Good to see you again."

And to reporters yet another political jab: "President Bok says this is one of Harvard's finest classes - and I never disagree with presidents."

After spending about half an hour at the reunion, Kennedy left with his wife, Joan, who had arrived earlier.

At an afternoon party, other classmates were drinking gin from plastic cups and reminiscing about past indiscretions.

There was May Day 1954, for example, when the two classmates had concocted a bathtub full of Moscow Mules - vodka mixed with ginger beer.

"That sure was on hell of a good party," said one. "We got at least a fifth of the college smashed that day - God, it was great."

The recipe for the brew had come from the students' proctor, now national security adviser Zbingniew Brzezinski. "He was in charge of making sure we didn't burn down the campus.

"Wonder what ever happened to him?" They laughed - not enviously but a laugh of equals. "Good ole Zbig."

About 600 of the nation's quintessential over-achievers were taking a break yesterday. After a quarter century of success, the Harvard class of '54 had come home.

The class that boasts Kennedy and John Updike also includes: 100 lawyers - most partners in firms - 92 doctors, 76 bankers and investors, 14 writers and nine clergymen, with nine unemployed or retired.

More than half the class earns over $50,000 a year, with 13.8 percent making over $100,000.

It was lunch time and the Harvard band was trooping around the Eliot House lawn blaring over the hundreds of alumni eating baked salmon smeared with mayonnaise and drinking bloody Marys and gin-and-tonic.

It was difficult to imagine that 25 years ago across the yard some of these pillars of the American power struture were embroiled in the "Great Pogo Riot." The creator of the comic strip "Pogo," Walt Kelly, was in town and several students decided to stage a demonstration - to run Pogo for president.

The spring weather, and abundance of beer in Harvard Yard and the president of policemen resulted in the reported beatings of many Harvard students.

"I remember running with the Aga Kahn (class of '58) and we ran into a very large cop," says movie actor Joseph Mitchell. "When I tried to get away, he booted me in the a - so hard that it hasn't been the same ever since."

Not surprisingly, it was a reunion with a litany of success stories. In the arts there were author Christopher Lasch, Boston Globe publisher William O. Taylor, and New York Museum of Modern Art director Richard Oldenburg.

In academics there were George D. Langdon Jr., president of Colgate College, John Richards II, dean of Phillips Academy and Michael V. Harrison, dean st Michigan State University.

In government there were Sen. John Culver (D-Iowa) and Reps. Anthony C. Beilenson (D-Calif.) and David R. Bowen (D-Miss.)

But above all, there was Edward Kennedy.

"President Carter has been a disaster for this country and I don't think our classmate's performance would be much better," one alumnus wrote in the class report. However, most of Kennedy's classmates - at least the Demoncrates - are some his strongest boosters.

"There is some resentment of Kennedy, but there is also tremendous pride," said a Time editor, Ronald Kriss. "They fell pleased that he is a member of the class but they don't necessarily want to see him president."

Said Boston attorney Arnold Slavet, "Ted Kennedy is a good example of what someone can do with talent, drive and a Harvard education."

Of course, Kennedy almost didn't get a Harvard education. In fact, he was forced to graduate two years after his classmates because of what many now refer to as a "hiatus" from Harvard.

"During the second semester of my freshman year, I made a mistake," Kennedy told reporters during his first senatorial campaign in March 1962. He was having trouble in a Spanish course and "I became so apprehensive that I arranged for a fellow freshman friend to take the examination for me."

He caught and expelled along with the friend. "What I did was wrong. I have regretted it ever since," said Kennedy during that heated campaign. "The unhappiness I caused my family and friends, even though 11 years ago, has been a bitter experience for me but it has also been a very valuable lesson.

"I worked hard, passed all my courses - some with honors - and I was graduated in good standing in 1956," said Kennedy, who excelled in public speaking and recieved honor grades in history and government.

The mid-'50s were a time of achievement and prosperity across the nation, and Harvard's best and brightest were no exception.

"We were fortunate to have spent our formative career years in a decade that extremely congenial to high motivation," said Sen. Culver, a former star fullback who helped turn Harvard's beleagured football team to a 6-2 winning season in his senior year.

"There was still a strong sense of promise and opportunity, and morale was high," we said.

Time Magazine dubbed it "the silent generation" - a label Kriss now calls "a canard."

"We were not as quiet was were sometimes made out to be" he said, "just more discreet about our likes and dislikes."

The class' biggest fear was Sen Joseph McCarthy, whose campaign against communism turned on Harvard, with charges that the faculty was "pink-tinged."

"We were outraged and amused by this kind of buffoon," recalls John Updike. "Nixon seemed that way too. From the safety of Harvard it looked like an aberration in American politics - a subject in which we had little interest."

Their interests have changed with their growing success. Even the less successful of the class of '54 are overachievers.

The Class includes a member of the du Pont family, Lammot du Pont Copeland Jr., famous for filing one of the largest personal bankruptcy claims in U.S. history in 1971. He now writes jokes for a living.

And there is Mitchell, the class orator, who says he will soon appear in a film called "Feedback." Mitchell says he spent two years in the federal penitentiary in Lewisburg, Pa., for selling LSD. His prison mate was Jimmy Hoffa.

"It's uncanny," says Mitchell, "my bust and Ted's Chapp bust abd Ted's Chappaquiddick came almost sumultaneiously." CAPTION: Picture, Joan and Edward Kennedy at last night's reunion; by UPI