Whether they realize it or not, the mayors of Chicago, Albany, N.Y., and New York City may well hold the key to African participation at the 1984 Los Angeles Summer Olympics.

These U.S. cities are the scheduled sites for a tour by a Soiuth African rugby team in September. It would not be an exaggeration to surmise that the same 41 African nations that boycotted the 1976 Montreal olympics because of this same rugby team may boycott the '84 Games if U.S. cities play host to a team representing racist South Africa. And there are sure to be more sympathizers this time among the world's Olympic committees.

Like a sore that won't heal, the question of South Africa's place in international sports surfaces again and again. Make no mistake, this team is the Springboks, for years the best rugby team in the world. But excellence aside, the overriding issue now is not Soiuth African prowess in scoring tries but whether the Springboks should be allowed to play within our borders.

I think not.

This South African team should not even be granted visas. They are coming to the U.S. as a national team -- not as a group of individuals. I would have nothing against individual South African athletes coming here as long as they disavow apartheid in writing.

In purely practical terms, if this rugby tour becomes a reality, there is the possibility of about 50 nations becoming no-shows in '84 in Los Angeles. But more immediately, any rugby match between a South African team and some American side could be played only under armed guard. Will the mayors of New York City, Albany and Chicago be willing to provide police protection for a team from an avowedly racist nation? What if the police refuse? Do you think any black policeman can be detailed for this assignment? Will any union airport baggage handlers transport their lugage?

Of course, the moral issue remains, as before, the telling argument. The Springboks currently are in New Zealand playing the All-Blacks (the nickname for the New Zealand rugby team). The New Zealand foreign minister, Brian Talboys, has likened his country's hosting the Springboks to "driving the wrong way on a one-way street."

He is right. If we hold apartheid to be morally repugnant, then can some representative team from South Africa be any less repugnant?

At present that one-way street heads away from South African participation in international team sports until their government-imposed system of apartheid is eradicated. The South Africans were recently admitted to pass through the United States on performance visas, which allowd them to work on a temporary basis. To allow them to play here in Spetember would be to signal other countries that it's okay to do the same. Australia already has refused to allow the Springboks evern to refuel on their way to New Zealand.

Ironically enough, the very idea of a sports boycott used to be considered in bad taste by the leading industrial nations. But President Carter's order to the U.S. Olympic Committee to boycott the 1980 Moscow Olympics for the Soviet invasion of Afghanistan changed all that. At one time 68 percent of us agreed with President Carter. And when the U.S. sets a precedent, others follow.

There are other premeditated precedents. The Gleneagles Agreement was signed by the Commonwealth prime ministers meeting in Scotland in 1977. These heads of government agreed to use their efforts to ban all sporting contacts with South African national teams. New Zealand already has broken the agreement.

South Africa now is banned in more than 15 major team sports competitions, including tennis, soccer, track and ield, as well as the Olympics. It's not for mundane reasons that this overseas tour by the Springboks in their first in five years. No country would have them before.

New Ork Mayor Ed Koch already is going the wrong way. He has said that "the rugby match between a South African team and a U.S. all-star team would be permitted on city facilities." That facility is Downing Stadium in New York City. But the problem should not have reached Koch's desk.

U.S. visas are granted selectively by the secretary of state. That is the way it should be. The issuance of performance visas to the South African rugby team adulterates and profanes the very standards by which we allow in our foreign visitors and friends. Secretary of State Alexander Haig should see to it that this team does not enter our borders.

It is quite proper for us as Americans to espouse freedom of speech and to hear out differing points of view. But to accommodate willingly a team of athletes from a country whose very laws are morally antithetical to the entire world is just plain wrong.

President Carter caused the USOC to boycott the Moscow Olympics because of State Haig should likewise embargo the Springboks. For, as a team, they are simply not welcome.