The outgoing American ambassador here is leaving Zimbabwe, frustrated over lack of communication between the two governments and recent anti-American diatribes that have hurt relations between Zimbabwe and its largest aid donor.

Ambassador David C. Miller Jr., a Reagan administration political appointee who is cutting short his tour of duty after two years, has refused to comment publicly on the reasons for his early departure.

But diplomatic colleagues and other knowledgeable sources say Miller has grown weary of seeking dialogue with a Third World government that is officially nonaligned but often appears to go out of its way to bait Washington even while it accepts millions in foreign aid. He is expected to recommend that the United States reevaluate its aid program here.

Earlier this week, Information Minister Nathan Shamuyarira accused the United States of providing funds and personnel to "Radio Truth," a clandestine station in South Africa that broadcasts antigovernment propaganda and support for rebel movements in Zimbabwe, Mozambique and Angola and that is widely believed to be controlled by South African military intelligence. The U.S. Embassy called the charges "preposterous."

Several weeks ago, Enos Nkala, the Cabinet minister in charge of police, accused the CIA of supporting dissident activities in southwestern Matabeleland and threatened to recommend the expulsion of American and British diplomats for repeating allegations by local human rights advocates that police have engaged in torture. Although some Zimbabwean officials have said privately that Nkala was not speaking for the government, it has not withdrawn his remarks despite a strong American protest.

Both Shamuyarira and Nkala are senior Cabinet members considered close to Prime Minister Robert Mugabe. He himself has not endorsed the allegations and has long described relations with the United States as "friendly" despite strong disagreements on issues like economic sanctions against South Africa and support for anti-Marxist rebels in Angola.

Zimbabwean sources contend that Miller, who served previously as ambassador to Tanzania, expected too much of Mugabe's inexperienced government, which came to power in 1980. They say Mugabe is aware of American anger and has directed subordinates to consult U.S. diplomats more regularly.

At the same time, they say, Mugabe has supported his foreign minister and United Nations ambassador in their often blistering attacks on American foreign policy.

Since the former white-ruled Rhodesia gained independence six years ago, the United States has provided $363 million in bilateral aid, making Zimbabwe one of the largest recipients of U.S. assistance in Africa.

The aid has gone mostly for development of schools, housing and health facilities and to provide scarce foreign exchange to Zimbabwe's robust private sector for essential imports. The program is considered by many aid experts to be one of the most successful on the subcontinent and has helped Zimbabwe maintain its mixed economy and multiracial society even while Mugabe has espoused Marxism as his ultimate political goal.

But American diplomats and lawmakers have come to question whether the program makes sense in a country that is not only one of Africa's most prosperous but also one of those most critical of U.S. policies. American hopes that Zimbabwe would emerge as a regional peace broker and an example of racial and tribal harmony to its white-ruled neighbor, South Africa, also have not materialized.

Partly as a result, American aid levels have dropped from a high of $75 million in 1982 to $40 million in 1984 and $21 million this year.

"There's a really classic disjuncture here," said a diplomatic source. "The U.S. keeps sending in the money and the Zimbabweans just keep whacking away at us. At this point it's worth sitting down to say, 'We've done what we set out to do; now to what end do we continue?' "

American and other western diplomats have been especially angered by what they see as a lack of responsiveness and professionalism in Zimbabwe's Foreign Ministry. They consider Foreign Minister Witness Mangwende an amateur who issues a stream of antiwestern pronouncements, yet seldom makes himself available for consultations.

One example often cited is a message last September from Secretary of State George P. Shultz to black leaders and governments in southern Africa soliciting comments on the administration's decision to change course and impose limited economic sanctions against South Africa.

All but two of the recipients responded, sources here said; one of the non-respondents was Zimbabwe's Foreign Ministry.

The Reagan administration also has been angered regularly by Zimbabwe's votes in the U.N. General Assembly and during its two-year stint on the Security Council. U.N. Ambassador Vernon Walters rated Zimbabwe the most hostile of all nonaligned black African states in a recent report.

Miller, who is set to leave here April 17, is quitting to form a venture capital firm based in Washington. He is a conservative Republican and close friend of Chester A. Crocker, the Reagan administration's top Africa envoy.

Miller came here in 1984 after three years as ambassador in Tanzania, where he won high marks for maintaining a dialogue with the socialist government of then-president Julius Nyerere, another outspoken critic of U.S. foreign policy.

Miller's replacement has not been named.

Three other senior American diplomats -- the embassy's second-ranking officer plus the two top officials of the Agency for International Development's office here -- are also leaving in the next few months, although American officials insist these moves are coincidental and do not reflect displeasure with Zimbabwe.

Miller reportedly came here with high hopes of establishing a personal rapport with Mugabe, but colleagues say he has found the Zimbabwean leader far less accessible and less interested in diplomatic give-and-take than Nyerere.

One exception was a 2 1/2-hour meeting last month between Mugabe and visiting U.S. envoy Frank Wisner, who is Crocker's chief deputy. Knowledgeable sources said the talks, which included four other U.S. and four other Zimbabwean officials, were candid but friendly, and Mugabe distanced himself from Nkala's CIA acusations and from Mangwende's snub of the Shultz letter.

The meeting ended with a Mugabe pledge to maintain better communications, but diplomats say they have detected no improvement since then.

Zimbabwean sources said that while the country remained grateful for U.S. aid, relations have chilled since late 1983 when the Reagan administration halved its aid, a move State Department sources said was partly to express anger with Zimbabwe's votes in the U.N. Security Council. Zimbabwean sources said a further cut in aid, were it linked to American displeasure, would only aggravate the situation.