A bomb planted inside a city bus killed five people and wounded 20 here yesterday. The force of the explosion tore off the back of the bus, causing damage and severing power lines in the Beit Vagan section of West Jerusalem near Mount Herzel.

One of the dead was identified as Richard Fishman, 30, a medical student from Silver Spring, Md. Five of the injured were in critical condition.

It is unlikely that Israel will retaliate across its borders even though the Palestine Liberation Organization in Beirut has claimed responsibility for the act. Unlike an attack in March, when a band of Palestinians crossed the coast in rubber boats and killed 35 Israelis on the main Haifa-Tel Aviv road, yesterday's bombing was committed by saboteurs within Israeli-controlled terrority.

In the past it has not been Israeli practice to retaliate into Arab countries when terrorist attacks have come from within Israeli territory. Several suspects have already been arrested.

The bus left the Arab section of East Jerusalem shortly after 1 p.m. and wound its way through the Jewish section of the city, picking up many Jews who were going home for the sabbath. The bomb, according to police experts, was apparently placed in an overhead rack near the rear of the bus.

Reporters who reached the scene described the debris of shopping bags and religious books in the streets of the Orthodox Jewish section where the bomb exploded near Yad Vashem, Israel's memorial to the six million Jews killed by Nazi Germany.

Although dozens of bombs have been left in buses in recent months throughout Israel, this was the worst such incident in Jerusalem since Feb. 14 when two people were killed and 35 injured in a similar bus bombing. On April 26 two German tourists were killed and five wounded when a bomb was tossed into a bus in the West Bank town of Nablus.

Often such bombings take place on Friday when the markets and buses are filled with Israelis preparing for the sabbath.

Mayor Teddy Kollek, interviewed on Israeli radio, said that although police and citizens were very alert, "You can't fight everything." Yesterday's incident was the kind of thing Israelis always feared would happen, "and then it does happen," he said.

The mayor said he thought the incident was connected with next Sunday's holiday marking the 11th anniversary of the reunification of Jerusalem following the capture of Arab East Jerusalem in the 1967 war.

The mayor said it was too early to say whether yesterday's incident would change plans for Sunday's celebration, but he said he would be against any chance.

"I would not want these bastards to let our festivities be in any way diverted," Kollek said. "That would be the success they would pray for."

News services reported the following details:

Police said the bomb was an 81mm mortar shell with a timing device.

The first rescuers were drivers from a nearby taxi stand who radioed for assistance and transported most of the wounded to hospitals.

"I saw 10 or 12 people on the seats and the floor," said taxi driver Yitzhak Bayanzi. "There was blood and severed limbs everywhere."

Yaacov Barashi, manager of the taxi stand, said a girl of about 17 was decapitated by the blast.

In other Middle East developments:

Palestinian leader Yasser Arafat and the commander of U.N. peacekeeping troops in Lebanon began talks in Beirut on the situation in southern Lebanon. At least 10 persons were killed there earlier this week in a flareup of fighting.

Gen. Emmanuel Erskine, the U.N. force's Ghanaian commander, said Arafat had agreed to allow peacekeeping officers to be based at Beaufort Castle, a major guerrila stronghold outside the area controlled by the U.S. force.

Lebanese regular troops manned positions in the north of the country in the first major deployment of such units since the Lebanese civil war ended in November, 1976. The move followed clashes Wednesday between rival rightist militiamen in the area.