The withdrawal of Sen. Edward M. Kennedy (D-Mass.) from the 1988 presidential race is seen by many Democrats as signaling the end of a cycle in their party's history and opening an era whose leadership and direction will be settled only by 30 months of battling.

Kennedy's unexpected announcement came 13 months after fellow-liberal Walter F. Mondale lost the presidency in a landslide and 13 months before a third nationally prominent liberal, House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.), is scheduled to retire.

Taken together, said Alvin From, executive director of the Democratic Leadership Council (DLC), a group of elected officials, the three events "will speed the generational change in the party and open the door for a new group of people to redefine the party and what it stands for."

That effort at redefinition began in 1984, with the "new ideas" challenge by Sen. Gary Hart (D-Colo.) to Mondale in the primaries. It has accelerated this year at meetings of the DLC and in other forums, and Kennedy appeared to many observers to be seeking opportunities to show that he was not a doctrinaire defender of past Democratic programs.

Nonetheless, Kennedy was linked by family name and a 23-year Senate voting record to the causes of disarmament, civil rights, national health insurance and other domestic welfare programs, just as Mondale was bound in the 1984 campaign to the policies and traditions of his mentors and benefactors, Vice President Hubert H. Humphrey and President Jimmy Carter.

Along with O'Neill, the three men symbolized to the public, more clearly than any other trio, traditions and policies of the Democratic Party going back at least to the 1960s.

That traditional Democratic Party is not dead, and its proponents remain prominent in the leadership of the House and Senate. In 1988 presidential politics, Jesse L. Jackson will certainly speak for the party's traditional concern for minorities and the poor, and New York Gov. Mario M. Cuomo is viewed by many in the leadership of organized labor as embodying traditions of the last Democratic New York governor who made it to the White House, Franklin D. Roosevelt.

Nonetheless, Ann Lewis, director of the liberal Americans for Democratic Action, said Kennedy's withdrawal "removes the most familiar figure for liberals and clearly is going to change the debate."

Hart and such other younger generation Democratic hopefuls as Sen. Joseph R. Biden Jr. (Del.), Rep. Richard A. Gephardt (Mo.) and Arizona Gov. Bruce E. Babbitt anticipated running a "future vs. past" campaign against the 53-year-old Kennedy.

Now, said Robert Shrum, a Kennedy speechwriter and political adviser, instead of Democratic aspirants explaining why "it shouldn't be him Kennedy , they'll have to start explaining why it should be them."

That shift is expected to be felt most acutely by Hart, who inherits the historically uncomfortable role of front-runner, and by Cuomo, perhaps the logical claimant to the equivalent title of "eloquent white liberal spokesman."

Several strategists said yesterday that they expected front-runnership to be more of a burden than a boon to the Coloradan. Hart's top assistant, William Dixon, insisted yesterday that Kennedy's withdrawal will have no effect on Hart's decision, to be announced on Jan. 4, on seeking a third Senate term.

Beyond short-term tactical decisions, all of the active and potential 1988 candidates face the challenge of sorting out their own positions and achieving their desired starting points for the coming contest.

Hart was wounded in 1984 when Mondale successfully suggested that there was little substance to the challenger's "new ideas" theme. As if to answer the taunting Mondale question, "Where's the beef?" Hart this year has made provocative proposals in areas ranging from trade policy to national service for young people and is working on an alternative to President Reagan's new budget.

Still, Hart's policy approach is far from clearly defined. On the Gramm-Rudman-Hollings balanced-budget act, when Kennedy zigged right to support the bill, Hart zagged left, denouncing the proposal as a threat to vital domestic and defense programs. Lewis speculated that Cuomo actually may find Hart a serious challenger for liberal support in 1988.

The New Yorker is best known to the public for his 1984 Democratic National Convention keynote speech, dwelling on the plight of the poor, the homeless and the elderly. Early this year, however, he defined himself as a "progressive pragmatist," and he is expected to use his reelection campaign next year to portray a record that includes tax cuts and business development incentives, as well as increases in education and welfare funding.

The others are known for particular issues, rather than a broad political philosophy. Biden was an ardent critic of busing and more recently has been identified through his committee assignments as a critic of the Reagan foreign policy and an opponent of Attorney General Edwin Meese III and his subordinates.

Gephardt was cosponsor with Sen. Bill Bradley (D-N.J.), another 1988 possibility, of the major Democratic tax-revision bill that helped frame the tax debate this year and a promoter of trade legislation which Hart, among others, labeled protectionist. Babbitt is perhaps the Democrats' leading advocate of the policy of returning responsibilities to the states and has gone beyond almost anyone else in suggesting means-testing for Social Security and other federal entitlement programs.

Babbitt and Gephardt are leaders of the DLC, which Jackson, another likely 1988 contender, sees as a group "trying to push the Democratic Party to the right."

Jackson said in an interview yesterday that Kennedy's withdrawal "removes one of the fingers in the dike to slow down the flood to the right . . . . With him gone, it obligates us in the Rainbow Coalition to fight even harder for that point of view."

Even as the old order passes, their tradition of battling each other seems certain to be kept alive by the new.