President Carter, who has relied relatively little on prepared speech texts for his public remarks, is about to lose two speechwriters in addition to his chief speechwriter, Jim Fallows.

Caryl Conner, who joined the White House staff last spring, said yesterday that she plans to resign about Jan. 1 to do freelance work and public television commentaries.

Meanwhile, Jerome Doolittle, a Carter speechwriter since the beginning of the administration, said he plans to leave the White House soon to work for another government agency.

Fallows, who announced his resignation several weeks ago, will leave the White House at the end of this week to become Washington correspondent for the Atlantic magazine.

Although White House officials insisted that neither Conner nor Doolittle was being forced out, the exodus follows by a few months the placing of the speechwriter office under the direction of the president's assistant for communications, Gerlad Rafshoon.

Officials said it had not been decided whether Fallows will be replaced with another "chief speechwriter" but it is widely assumed that Rafshoon's deput, Grey Schneiders, will take over day-to-day control of the speechwriting apparatus.

Fallows, who said from the first that he intended his stay at the White House to be about two years, suggested yesterday that his resignation may have played a role in the others' decisions.

"The people here are all people I brought in" he said. "I decided it was the right time to go and independently others may have reached the same conclusion."

Conner said she was leaving the White House with "enormous respect" for the president Doolittle, who among his other duties, wrote jokes for the president was known to be dissatisfied with the White House work.

Carter has written some of his own speeches - for example, his pronouncement on Soviet-American relations delivered at the Naval Academy this year - and frequently speaks from "talking points" rather than prepared texts. His speechwriters have complained privately about their lack of access to the president and about interference in their work by more senior presidential aides.