The leaders of the three major nonwhite organizations in South Africa pushed ahead this week on a course likely to lead to a confrontation with the government.

The three met in Cape Town Monday for the first executive session of an alliance of Zulus, South Africa's largest black tribe, with the country's Indian and colored (mixed race) populations.

The executive session christened their grouping the South African Black Alliance and began preparation for a convention to write a nonracial constitution for South Africa.

The alliance is being organized in defiance of South African law, which prohibits groupings across racial lines. The white-controlled government has sought to keep the races separate by establishing nine quasi-independent "homelands" for blacks and creating separate parliaments for whites, coloreds and Indians.

The chairman of the alliance, Zulu Chief Gatsha Buthelezi, said after Monday's meeting that the goverment of Prime Minister John Vorster would have to negotiate with it just as Prime Minister Ian Smith, leader of the white minority government in Rhodesia, was forced to bargain with moderate blacks in that country.

Buthelezi was joined at the meeting by Sonny Leon, leader of the colored Labor Party, and Y.S. Chinsamy of the Indian Reform Party. All three have extensive, though not total support from their racial groups.

Leon told reporters that the alliance is not yet ready to accept any white organization although it is not antiwhite. Acceptance of white groups would come after the alliance has estalished itself, he said.

The alliance faces opposition from young, mostly urban, radical blacks who argue that a violent confrontation with South Africa's whites is inevitable. Last Saturday a group of militants fored Buthelezi and Leon to leave funeral services for black nationalist leader Robert Sobukwe.

The populations of the Zulu, colored and Indian groups in South Africa is 5 million, 2.4 million and 750,000 respectively. Whites number about 4.5 million.