SECURITY - which is to say the control of the armed forces - is the key to the fragile peace negotiations now under way in Rhodesia. it could hardly be otherwise after years of spreading guerrilla warfare. The whites, outnumbered 20 to 1 by black Rhodesians, fear that majority rule would mean pillage and seizures of property. Blacks assume that anything short of majority rule would simply mean continued white dominance. The British government, with the active support of the United States, has now published the outline of an extremely complex peace plan. Nobody can know at this point whether either side of the Rhodesian war will accept it. But the striking thing is that no one, so far, has flatly rejected it.

It is an intellectual challenge of considerable dimensions: how to create a new government strong enough to keep the peace without giving either side total sway over the other. The British solution begins by asking Prime Minister Ian Smith and his government to step down and return the country to the colonial status against which it rebelled 12 years ago.

Britain would then appoint a commissioner to run the country for a few month while elections were held under universalsuffrage. The country would then become independent as the Republic of Zimbabwe, Under a constitution that is part of the deal.

There would be a sweeping bill of rights, but laws do not enforce themselves. The question keeps coming back to the nature of the future Zimbabwe National Army. The British Foreign Secretary, David Owen, states that this army would be "based on the liberation forces," although it would also include "acceptable elements" of the present white government's armed forces. "It is self-evident that this army must be loyal to whoever is elected president and whoever forms the new government of Zimbabwe," said Mr. Owen Loyalty, unfortunately, is hard for outsiders to gurantee.

But for all of its obvious perils, this proposal enjoys one very great advantage: All of the alternatives are far more perilous. Neither side can expect a quick military victory, and both now understand the costs of prolonged fighting. The dismaying example of Angola is pushing everyone in the direction of realism. The black forces finally won in Angola, but the various factions of those armies are now at war with each other. The presence of the Cuban contingents there provides a reminder to Africans of the temptation for other powers to fish in troubled waters. Meanwhile, the exodus of the terrified Portuguese has all but destroued Angola's economy. Last year, for Rhodesia, the United States suggested an international fund to buy out the whites. There is a fund in the present British plan, but it offers the present residents incentives to stay.

The plan is the result of an extraordinary diplomatic exercise by the British and U.S. governments. The American ambassador to the United Nations, Andrew Young, joined Mr. Owen in the long tour through African capitals - explaining, persuading, recruiting support - that finally ended in the Salisbury talks with Prime Minister Smith. Mr. Smith has applied a lot of unflattering adjectives to the plan, but, apparently, he is prepared to consider it a bit further.

If this plan collapses, the result will be bloodshed and desolation on a widening scale. But if it holds up, Zimbabwe's neighbors and a good many other influential countries will share a degree of responsibility for peace and stability there. That is an extraordinary asset for a new nation. it is still far from clear that a peaceful settlement is possible in Rhodesia. But the best hope for it - and perhaps the only hope - lies in the proposal that Mr.Owen and Ambassador Young have carried to Salisbury.