Former president Jimmy Carter will write a memoir about his four years in office, and has chosen New York literary agents Marvin Josephson and Lynn Nesbit to represent him, according to Josephson and Carter's lawyer, Lloyd Cutler.

The agents will also represent Rosalynn Carter if she decides to write her memoirs, a project she reportedly is considering.

The choice of agents was made late last week, according to Cutler, who along with Atlanta attorney Charles Kirbo, a friend of Carter's, interviewed six or seven agents.

"There were several we were very impressed with," Cutler said yesterday. "But we decided the best opportunity was with Marvin Josephson and Lynn Nesbit."

Josephson also represents Henry Kissinger, and handled the 1977 deal in which Kissinger received a reported $2 million for rights to his memoirs.Josephson, who also represented actor Steve McQueen before his death last year, is head of Marvin Josephson Associates, the parent company of International Creative Management (ICM), a large agency handling 2,400 clients. Nesbit is the senior vice president of the literary department of ICM.

Cutler said that about 10 different inquiries from various agents were received. In making the decision, the lawyers talked to the agents as well as to their clients. "It's one of those comparative judgments," said Cutler, "like choosing a lawyer or an underwriter."

Among the agents reportedly interested in representing Carter were Norman Brokaw, Morton Janklow, Scott Meredith and Irving Lazar -- some of the biggest names in the literary and talent business.

"I did have a meeting with Charles Kirbo" to present an "international" package deal, Brokaw said yesterday. "A lot of people actively pursued [the book contract]. I didn't." Brokaw also handles President Reagan's daughter, actress Patti Davis, as well as former State Department spokesman Hodding Carter and former president Gerald Ford. Asked if the lineup of Republicans hurt his chances, Brokaw said, "I don't think so." And Brokaw discounted any possible prejudice against him stemming from client Hodding Carter's critical remarks about Jimmy Carter in the February issue of Playboy magazine.

Josephson and Nesbit met with the Carters at the White House in early January, and "the president liked them both," said Cutler.

Josephson, reached yesterday by phone in Aspen where he is vacationing, said he had no idea how much his connection with Kissinger helped or hindered the selection. "This has nothing to do with ideology," he said.

At this point the agents do not know what the former president plans to write. "I know he has extensive notes from four years and documents," said Josephson. "When you talk to him one-on-one, you know this man is capable of writing a powerful book. I think he plans a candid and forthright memoir, but I have no knowledge of specifics."

Carter is not currently working on his memoirs, and is expected to leave today for a vacation in the Virgin Islands. Former White House press secretary Jody Powell said yesterday, "It's the first time in 15 or 20 years he will be able to go and stay and come back whenever he wants to."

Josephson does not expect Carter to have a ghostwriter for the book, a common practice among noted officials and celebrities. "I think he plans to write it himself," said Josephson. As for Rosalynn Carter, "I think she's a fascinating woman," he said, "and would have an interesting book to write. I hope she will write it."

Bidding on the memoirs has not begun, and neither Cutler nor Josephson would discuss what kind of financial arrangement they expected to make with a publisher. In the past few years, former presidents and leading administration officials have drawn at least six-figure contracts.

"Every important publisher in America is interested," said Josephson. "It will be at least a couple of weeks before we report back. At some point there will be a meeting of the minds between Carter and publishers."