The Cambodian government, which has frequently been accused of killing hundreds of thousands of its own people, yesterday offered to open its borders to Western observers to counter charges of massive human rights violations.

The offer was made by Ieng Sary, Cambodian foreign minister, who said that U.N. Secretary General Kurt Waldheim has been invited to visit Democratic Kampuchea, as Cambodia has been known since the Communists gained power three years ago, to "see with his own eyes" whether the charges are valid.

The Cambodians have repeatedly denied these charges, and yesterday the foreign minister said that the Cambodian government may allow American journalists to visit Cambodia later this year.

A spokesman for Waldheim said the secretary had not decided whether he will accept the Cambodian invitation. The spokesman cited Waldheim's heavy schedule and his refusal to travel during the General Assembly session that ends in December.

The new willingness of the Cambodians to talk freely about their country seems to stem from the strong international criticism that has been directed during the past year at the extreme policies of Cambodia.

The Vietnamese have accused Cambodia of fomenting their months-long border war and have invited Western journalists to the Vietnamese front. Yesterday, Ieng Sary said the Vietnamese started the war.

"We are inviting you journalists to our country not to mobilize opinion against Vietnam," Ieng Sary contended yesterday. "We want you to see for yourself whether there are human rights violations."

Documents submitted to a U.N. subcommittee by the Human Rights Commission, the United States, Britain and Canada have accused Cambodia of violating almost every article of the Universal Declaration of Human Rights adopted by the United Nations in 1948.

Estimates of the number of Cambodians who have died because of starvation, overwork and executions range from hundreds of thousands to more than 3 million and are based on refugee accounts. Ieng Sary denied such charges yesterday but said that some Cambodians have been killed because they were "infiltrators and agents of the Vietnamese or U.S. imperialists."

Amnesty International, one of the leading proponents of human rights activism in the world, applauded the Cambodian announcement.

"It is a very welcome invitation to the United Nations to visit Democratic Kampuchea," a spokesman said. "It is a very positive response to international expressions of concern and an important recognition of the role of the U.N. in human rights protection."

The traveling minister for the Cambodian government also admitted that there had been problems in 1975 and 1976 but said these had been solved. The main problem facing Cambodia, be contended, was the threat of Vietnamese intervention.

Yet Ieng Sary promised in a speech delivered Thursday to the General Assembly that Cambodia "will sign a treaty (of friendship with Vietnam) whether in Phnom Penh, or in Hanoi, or elsewhere, if Vietnam show that it really wants to have relations of friendship with Kampuchea."

At his press conference yesterday, the Cambodian minister made a similar offer to the United States and France, saying his country wants relations with both these Western nations and without conditions.

When asked about Norodom Sihanouk, the prince who led Cambodia until he was overthrown in 1970, Ieng Sary distributed photographs he said were taken last month at an official Phnom Penh dinner party. They show Sihanouk and his wife Monique being entertained by Khieu Samphan, one of the top leaders of the country. They the first photographs of the prince made public since his retirement three years ago.

"He is living now in his palace," said Ieng Sary. "He is living now like before, comfortably."

While the Cambodian minister refused to answer direct questions about human rights issues, he was expansive on the conflict between his nation and Vietnam.

"This border problem is' a small problem," he said. The real problem is that Vietnam wants to swallow all of Kampuchea."

At a reception given by the Cambodian U.N. delegation Thursday evening, a Chinese spokesman added a big-power viewpoint on the conflict in the region.

"What did I like about the Cambodian speech?" He asked rhetorically. "The paragraphs about Vietnam."

The Vietnamese are allied with the Soviet Union while the Cambodians count the Chinese as the strongest of their few friends.

Ieng Sary said he wanted to change Cambodia's isolation as well. Relations have been established with Thailand, Malaysia, Singapore and Indonesia and he plans to leave next week for the Philippines to seek normal relations there.

The foreign minister would make only one comment about U.S. criticisim of his regime and that was an answer to Sen. George McGovern (D-S-D.), who recently called for international intervention against Cambodia.

"McGovern, I think he is dreaming, that he just awoke from a long dream," Ieng Sary said. "There are many people who think we will not succeed but we will. McGovern has never known a devastating war in his country. He has no idea what problems there are after a devastating war."