A high State Department official expressed confidence yesterday that Rhodesian Prime Minister Ian Smith will agree to discussions initiated by the United Nations on ending the civil war in his country.

Richard M. Moose Jr., assistant secretary of state for African affairs, told reporters that although Smith has not approved the Anglo-American plan for a peaceful settlement of the dispute, "we believe he'll be prepared to enter discussions" on a crease-fire.

Moose's comments were the most optimistic so far from the administration since Smith all but rejected the British-U.S. proposals offered two weeks ago as "crazy" and "insane."

The assistant secretary declined to disclose the basis for his optimism, but it was understood that negotiators have had further discussions with Smith since British Foreign Secretary.

David Owen and U.S. Ambassador to the United Nations Andrew Young met with Smith Sept. 1.

Under the proposals they presented to Smith, the U.N. Security Council will be asked next week to allow the secretary general to appoint a special representative to help arrange a cease-fire. The representative would work with British Field Marshal Lord Carver, who would be Britain's resident commissioner.

They would meet with military commanders from Smith's side and from the black nationlists' side that is trying to overthrow Smith's white minority government.

If a cease-fire is arranged, a U.N. peacekeeping force would be introduced and the negotiators would next try to arrange a peaceful "surrender" of power by Smith. The prime minister has called the plan a "cunning scheme" to turn over power to the Patriotic Front, the main black nationalist group fighting the Smith regime.

Moose said that if Smith "wants to give this plan a fair shake," he will allow his military commanders to discuss peacekeeping arrangements with the U.N. and British representatives. The assistant Secretary also maintained that Smith has not rejected the Anglo-American plan "in toto."

Moose was also asked about the 10,000 to 15,000 combat troops that Cuba has in Angola. "We don't believe the Cubans have any business there," he said; "but it's of a burning issue."

He said the "main policy" of the United States is to seek an end to the struggle between two black nationalist factions in the southwest African nation.

The State Department has not yet raised the troop issue with Cuba but intends to do so, a spokesman said.