UNESCO's beleaguered director general, Amadou Mahtar Mbow, ruled out today any thought of resigning his post to prevent the United States and other western countries from leaving the organization.

In a rare interview, Mbow conceded that UNESCO -- the United Nations Educational, Scientific and Cultural Organization -- might have to trim its programs in the event of a U.S. withdrawal at the end of this year. He added, however, that the 161-member organization would continue to play a useful role with or without the United States.

The Senegalese director general described as "regrettable" a recent decision by Britain to follow the United States in giving notice of its intention to leave UNESCO. But he gave no indication that the departure of either country would have any effect on the way he runs the organization.

Western diplomats and UNESCO officials expect that the Reagan administration soon will confirm its decision to withdraw as of Jan. 1, 1985. The United States has accused the agency of mismanagement, extravagance and political bias in many programs.

While acknowledging that UNESCO has taken some steps to reform itself since the U.S. decision was announced, U.S. officials have depicted these steps as insufficient to warrant reconsideration.

While the United States has refrained from criticizing Mbow in public, the possibility of his resignation has been raised privately by some western diplomats and disgruntled UNESCO staff members as a way of preventing more defections and giving the organization a fresh start.

Mbow, who has headed the organization for 10 years, made clear today that there was absolutely no chance of his stepping down to appease his critics.

"I am convinced that my resignation would not help the organization overcome its problems," he said. "The director of an international organization cannot give in to pressures like this."

Asked how UNESCO would cope financially in the event of the withdrawal of the United States, which contributes a quarter of the organization's budget, Mbow said that there were two alternatives: either it would have to cut back its programs or it would try to persuade members to contribute extra funds on a voluntary basis.

He said that he would do his utmost to protect the present programs but that it would be up to the agency's executive board, which meets in March, to make the final decision.

Mbow, a former teacher and education minister in Senegal, appeared to reject a Soviet call for American staff members of UNESCO to be dismissed after a U.S. withdrawal.

Asked about the position of the 82 American citizens who hold professional posts in UNESCO, Mbow said he considered them "part of the international staff" and, as such, unaffected by official U.S. actions.