The Carter White House and Sen. Edward M. Kennedy (D-Mass) plunged into another round of backstage recriminations yesterday over the president's rejection of former Watergate special prosecutor Archibald Cox for a federal appeals court judgeship.

Kennedy allies construed the rejection of Cox -- who was the unanimous choice of a presidentially appointed judicial selection panel -- as a political slap at Carter's potential presidential opponent.

White House sources, in turn, insisted that Cox was rejected solely because his age -- 67 -- exceeded American Bar Association guidelines. The Carter aides accused Kennedy himself of playing politics by exerting "intense pressure" on behalf of his friend, Harvard professor Cox.

Kennedy is angry enough, sources say, to consider holding up in his Judiciary Committee appointment of whomever the White House picks. That, chuckled one White House aide, may create its own embarrassment for the Massachusetts liberal, since sources say Carter now favors Miriam Naveira, a Puerto Rican woman, also recommended by the panel, for the prestigious position on the 1st U.S. Circuit Court of Appeals.

"Political differences with senators are not among the factors involved in choosing a circuit court judge," said presidential spokesman Jody Powell. "Affirmative action is a consideration as long as the person is qualified."

Powell's remarks were the only on-the-record words from either side in a full day of strictly "for background" exchanges.

Both sides have in the past made much of the recent judicial reforms under which circuit court judceships were supposed to have been insulated from political pressures and awarded strictly on merit.

"All federal judges and prosecutors should be appointed strictly on the basis of merit without any consideration of political aspects of influence," Carter declared during his 1976 presidential campaign.

The Carter-appointed judicial selection panel for the 1st Circuit -- which includes Kennedy's home state, the rest of New England and Puerto Rico -- departed from standard practice first by ranking its recommendations to the president. The committee placed Cox, Kennedy's choice, at the top and Miriam Naveira, fifth. Guidelines call for submission of five qualified candidates -- period.

The White House, or its part, took advantage of the bar association's recommended age limit -- 65 -- to knock Kennedy's man off the list entirely, sources said, even though there was virtually no chance that the Judiciary Committee, which Kennedy chairs, would have let age block Cox's confirmation.

Meanwhile, both sides hotly deny that political factors are playing any role in their own actions, first publicized Thursday in the Boston Globe, Kennedy's hometown paper,

White House partisans said that Kennedy or his aides exhibited what they regard as inappropriately intense interest in the Cox appointment for weeks, even going so far as to imply that Cox's rejection might prompt retaliatory delays of Carter measures or appointments in Kennedy's Judiciary Committee.

Denying this, Kennedy supporters said that only politics could have prompted Carter to reject a man as eminent as Cox in the face of the unanimous recommendation of the selection panel.

Circuit court judges rank next to the Supreme Court justices in authority and prestige.

While senators are still actively involved in the dispensing of District Court judgeships -- one level below the circuit court -- the appeals-level appointments were to have been an affair strictly among the president, his selection panels and, ultimately, the Judiciary Committee. CAPTION: Picture, ARCHIBALD COX . . . Selection panel's first choice