South African Prime Minister P.W. Botha held nearly five hours of talks yesterday with Prime Minister Margaret Thatcher, who told him his white-minority government's policy of racial separation was "unacceptable."

The meeting was held at Chequers, Thatcher's country residence, while some 15,000 people staged a demonstration in London protesting Botha's visit, The Associated Press reported.

In a brief departure statement before flying to Switzerland to continue his West European tour, Botha said, "Each side put its views about the situation in South Africa quite candidly."

British government sources said that in an hour-long meeting alone with Botha before lunch, Thatcher told him South Africa's apartheid policy was not acceptable.

She also refused to sell South Africa reconnaissance aircraft when the subject "cropped up," said the British sources, who declined to be identified. The United Nations has imposed an embargo on the sale of arms to South Africa.

The one-day visit by Botha, 68, was the first by a South African leader since 1961. Botha spent nearly five hours at Chequers.

In a brief departure statement before flying on to Switzerland, Botha reiterated South Africa's demand that "foreign forces"--a reference to Cuban troops in Angola--withdraw from southern Africa as a condition for giving independence to the South African-ruled territory of Namibia.

He said he and Thatcher both "expressed their desire to help Namibia achieve its independence as soon as possible under peaceful conditions."

There were no smiles or handshakes as Botha and Thatcher posed for photographs at Chequers, and the statement left out the customary diplomatic pleasantries.

Thatcher had defended her invitation to Botha as a way of encouraging reform in South Africa, a major British trading partner and one-time colony.

Botha was on the third leg of an eight-nation West European tour.