In what is believed to be an unprecedented judgment in South Africa, magistrate I. J. J. Luther acquitted four young blacks today of subversion charges because he found that security police had coerced state witnesses into giving testimony.

The nine-month trial in Kempton Park, near here, included evidence of assaults and threats against witnesses and the testimony of a security police agent who admitted he had been told to fabricate evidence.

Although there have been many such allegations of coercion by security police in the past, this is believed to be the first time the charges have been upheld in a judgment resulting in acquittal.

Luther said he doubted the testimony of all the key state witnesses, and found that in three instances it was unacceptable as evidence because the witnesses had been frightened and threatened by the security police.

The four accused, all charged with furthering the aims of banned black student organizations, were Stanley Radebe, 27, Mthuthuzeli Madalen, 24, Lebona Ernest Mohakala, 23, and a woman, Nonkukuleko Innocentia Mazibuko, 23.

Mohakala also was charged with undergoing military training in Lebanon.

The trial received little publicity until its final day. David Soggot, lawyer for two of the accused, presented a document he termed "a review of the interrogational process according to the state witnesses."

In it he noted that six witnesses had said they were assaulted, some of them seriously, and five said they had been threatened with assault.

Nine had said the security police had told them what statements were required from them.

Five had said they were kept in isolation before and during interrogation.

Two had said they were told to fabricate evidence. One of these was a black, Portuguese-speaking Mozambican named Jim Kelly, who confessed under cross-examination that he was an agent of the South African security police.

Luther did not deal specifically with all these allegations in his judgment, but he indicated that he considered the three cases he cited as sufficient to discredit the whole state case.

The case also featured a trial within a trial when Soggot challenged the admissibility of a confession allegedly made by Radebe. Luther found that Radebe had been treated "in an inhuman fashion" and ruled the confession inadmissible.

Luther's judgments come only six weeks after a provincial supreme court judge in Pretoria found that a statement by Cedric Mayson, a Methodist minister charged with treason under the stringent security laws, was inadmissible as evidence because he had made it under duress.

In that case Mayson told Judge Pieter van der Walt that he had made the statement while being tortured by the security police. Mayson's trial is scheduled to continue April 18.

John Dugard, professor of law at Johannesburg's Witwatersrand University, said today he hoped the two findings meant the courts were beginning to take a more critical look at security police methods.

Magistrates are state employes and generally regarded as less independent than judges, who have tenure. Judges hear cases in the provincial supreme courts and the court of appeals, while magistrates sit in the lower courts--but often with extended jurisdiction that enables them to jail prisoners for up to 10 years.