President Carter said yesterday he expected diplomatic ties between the United States and Cuba to improve in the next few weeks, and State Department officials travelling with the First Lady said he was talking about an exchange of diplomats.

In Brunswick, Ga., shortly after seeing Rosalynn Carter off on a 12-day tour of Latin America, Carter told reporters, ". . . I think we will have indications in the next few weeks of strengthened diplomatic relations with Cuba, far short of recognition."

Although the President said he did not know "for sure"what those developments would be, Assistant Secretary of State Terence A. Todman disclosed in Jamaica yesterday that the United States has proposed to Cuba that the two countries exchange diplomats in what are called "interest sections." That means each country would place diplomats - but not ambassadors - in a neutral embassy in the other's capital. It is a step short of full diplomatic recognition, and one of the United States has taken with other countries.

Other U.S. aides in Jamaica on the first leg of Mrs. Carter's trip said the United States made the proposal in Havana May 10 and that the Cubans are very interested in it.

It was understood that Todman first mentioned the "interest sections" idea to Pelegrin Torres, Cuba's vice foreign minister, when they were negotiating on a fisheries agreement in New York in April. Torres made no comment, the officials said, but when Todman went to Havana May 1, the Cuban foreign minister, Isodora Malmierca, told Todman he woulk like to see a written proposal. The United States presented a paper on the subject May 10.

The U.S. Officials said the next step is to agree on how many diplomats will serve in the interest sections. Present U.S. thinking is to have eight to 10 American diplomats forming a section in the Swiss embassy in Havana and an equal number of Cuban diplomats working in the Czechoslovakian embassy in Washington. One U.S. official said that their duties officially would be to handle consular, legal, and welfare cases, but that, like Americans in the liaison office in Peking, such diplomats can handle all kinds of functions.

Yesterday's developments confirmed a remark that Carter made to out-of-town editors at the White House May 20. The President said then, "My guess is that in the near future we will have some diplomatic officials in Cuba and some Cuban diplomatic officials in Washington . . . as observers."

Carter said in Georgia yesterday he hoped the new developments would be similar to the ". . . small steps toward an increased ability to communicate and to discuss mutual concerns . . ." represented by fishing and maritime agreements negotiated with Cuba last month.

Today, as he has in the past, Carter also described the presence of Cuban military personnel in Africa, saying, "Obviously it would be better for the peace of Africa if other nations would not send troops and military forces into Africa . . . we would like very much for Cuba to refrain from this intrustion into African affairs in a military way."

But the President went on to say, "We see, though, that it would be better for our hemisphere if Cuba did have good relations with the other nations here.This is something that we hope to see in the future . . . I think we have demonstrated an ability to work with Cuba on the fisheries agreement and also on the maritime agreement."

It would be "a mistake to be too optimistic about it," he said.

Carter also said he still hopes to hold talks with Soviet leader Leonid I. Brezhnev in late September on limiting the use of strategic arms.

Asked to comment on Brezhnev's pessimistic assessment on stragetic arms limitations negotiations, Carter said he still believes that the last meeting between Secretary of State Cyrus R. Vance and Soviet Foreign Minister Andrei Gromyko in Geneva two weeks ago was "a great step forward."

We felt, for a change, that they genuinely wanted to have discussions leading to an agreement," Carter said. "They didn't exhibit this inclination when we sent Cy Vance to Moscow [In March]. So in that way it was an improvement. But it again would be a mistake to underestimate the great differences that exist between us."

He said he thinks ". . . we would certainly keep that hope [of a meeting] alive when we see how Gromyko and Vance get along at their next meeting . . ." There will be two more meetings between Vance and Gromyko before mid-September, Carter has said, probably in Geneva, but dates have not been set.

The President laughed and said, "I don't know," when asked how he and his nine-year-old daughter Amy would get along with Mrs. Carter away. "I feel lonesome already," he said ". . . I don't like for Rosalynn to be gone."

On the way back from the airport where he saw her off, to the retreat where he is staying on this resort island, Carter stopped his motorcade to greet about 50 children waiting beside the road for him to pass.

Another 200 or so youngsters lining the highway a short distance away then started running to him, yelling in unison, "He stopped! He stopped!" The President shook a few of their hands, laughed a lot, chatted with them for three minutes, then said, "I think we've got to go because we've got traffic held up all the way to Savannah," which is more than an hour away by car. State police close highways to all traffic as the presidential motorcade passes by.

Later in the day the President took Amy swimming at a home owned by Office of Management and Budget Director Bert Lance on nearby Sea Island.

Lance, who arrived Sunday, has played tennis and talked with Carter. But he said the conversations had nothing to do with presidential business.