President Carter has decided to challenge Sen. Edward M. Kennedy, (D-Mass.) to a series of nationally televised debates during their looming battle for the 1980 Democratic presidential nomination, according to White House aides.

If Kennedy agrees, it would be the first time that a president has met a challenger from the same party in such a forum.

In 1976 President Ford debated Carter, his Democratic opponent.

"The president has decided to debate Kennedy," presidential adviser said. "It will probably be in the same format of the debates in the 1976 campaign."

Under those ground rules, the League of Women Voters arranges the debates and selects questioners from print and broadcast journalists. Participants are permitted to offer rebuttals to the answers of their opponent, in keeping with traditional debating rules.

White House press secretary Jody Powell said yesterday that he is unaware of any plans to challenge Kennedy to debate. He did not deny that such a challenge may be issued, but said he knew of no "serious discussions" of it at the White House $.)

"It's possible," a spokesman for Kennedy said when asked if the senator would accept a Carter offer to debate. But Tom Southwick, Kennedy's press secretary, said no formal proposal had been made.

A drawback for Carter in offering to debate Kennedy is that the senator would be given instant equality with the president, erasing some of Carter's advantage as the incumbent.

That was the principal lesson of Sen. John F. Kennedy's 1960 debate with Vice President Nixon. The relatively lillle-known Democratic senator was suddenly an equal to the Republican who had spent two terms in the White House operation.

Clearly, Carter sees more potential gain than loss in meeting Kennedy head-on. Aides say the president has demonstrated an ability to deal with tough questions under pressure both at news conferences and town meeting. "We have an advantage over Kennedy in that respect," a Carter adviser said.

While Kennedy can be dramatic in delivering a prepared speech, he tends to speak in fractured phrases during the give-and-take of Senate debate.

If Kennedy accepts a challenge to debate, he too, will be violating an important political law: never debate an opponent when you are leading in the polls.