The election of the next U.N. secretary general was deadlocked today after four ballots in the Security Council when each of the two candidates was vetoed. Incumbent Kurt Waldheim was blocked by China, and Tanzanian Foreign Minister Salim A. Salim by the United States.

Diplomats from the West and the Third World gave Waldheim the edge when voting resumes Wednesday morning in the 15-nation council. Salim, who started out ahead with 11 votes on the first ballot, saw his backing erode rapidly, and ended up with only six votes, virtually ending his chance of election.

The only thing that can stop Waldheim from winning an unprecedented third five-year term is the overnight emergence of a compromise candidate who can command majority support.

There are several Latin American candidates who may come forward, but so far none has generated the broad backing that Salim had from the United Nations' Third World majority.

Without a viable alternative from among the developing nations, China is expected to be hard put to sustain its veto of Waldheim much longer.

Today's voting occurred behind closed doors and by secret ballot, but soon became general knowledge when the council members briefed scores of diplomatic colleagues clustered outside the roped-off chamber. The council voted on each candidate separately in each of four rounds, and each member nation was able to vote for more than one candidate.

The vote on Salim in the first round was 11-2, with two abstentions. One of the "no" votes was by a permanent member of the council, and therefore a veto. When Soviet Ambassador Oleg Troyanovsky said he had not cast it, it quickly became clear that Washington had decided to block Salim as expected.

The first round vote on Waldheim was 10-4, with one abstention. One of those "no" votes was the Chinese veto. It takes nine votes, and the absence of a veto by one of the council's five permanent members, to win the council's blessing for the top U.N. job, which pays $158,000 a year.

After the first round, China and the United States apparently maintained their respective vetoes and Waldheim sustained his 10 supporters, but Salim's backing dropped to 10, then eight, and finally six, as the support he had initially received from Britain, France, Japan, Spain and Ireland slipped away.

Some diplomats had expected China to drop its veto of Waldheim after Salim slipped below nine, but this did not happen.

Peking's stamina made it clear that no further purpose would be served by a fifth ballot today.

This encouraged the dark horses to emerge, among them Javier Perez de Cuellar of Peru, Carlos Ortiz de Rosas of Argentina and Shridath (Sonny) Ramphal of Guyana, who heads the Commonwealth.

There may also be a Mexican candidate before the battle is over.

The lack of support for Salim made it unlikely, Third World diplomats said, that the 156-nation General Assembly, which must approve the council's nominee, will seek to confront the big powers by rejecting Waldheim.