Law enforcement officials broadened their search for the suspected killer of a leading critic of Ayatollah Ruhollah Khomeini to include several states, the Caribbean and Canada yesterday after being convinced that the suspect is not hiding in the Iranian diplomatic headquarters here.

The Federal Bureau of Investigation had staked out the Iranian Interests Section of the Algerian Consulate on Wisconsin Avenue NW Friday to determine if the suspect, Daoud Salahuddin, was there. FBI officials said they not believe Salahuddin had not visited the section since Tuesday's assassination of Iranian exile leader Ali Akbar Tabatabai in Bethesda.

Meanwhile, D.C. and U.S. Park Police braced for today's anti-Khomeini demonstration in which about 3,000 to 7,000 Iranians are expected to participate.

Police said they were prepared for the possibility that pro-Khomeini groups may attempt to disrupt the 1 p.m. march from the Capitol to the White House and have assigned several hundered officers to patrol the demonstration route.

Tabatabai was busy planning the demonstration when he was gunned down in his Bethesda home Tuesday, and friends and associates, who gathered at the home yesterday to complete plans for the rally, say that it was because of such activities that he was killed.

In a related development, Ali Agah, the head of the Iranian diplomatic delegation expelled last April by President Carter, returned to the United States last week to become the new Iranian representative to the United Nations.

Associates say Salahuddin -- also known as David Belfield -- was drawn into a small circle of black American Muslims who have actively worked in support of the Islamic Republic by Agah, who often lectured blacks here on Islam.

It was during Agah's tenure here that Salahuddin and others in the group served as guards at the Iranian Embassy on Massachusetts Avenue NW. g

Law enforcement officials said they still are not sure of a motive for the killing, but prosecutors alleged in court that it may have been political.

Two men have already been arrested and charged in the case. Tyrone A. Frazier, 31, is accused of renting his Postal Service jeep to Salahuddin. A witness to the killing said a man dressed in a postal uniform gained entrance to Tabatabai's house at 9313 Friars Rd., Bethesda, by saying he had two special delivery packages that Tabatabai had to sign for personally.

A third person, Horace Anthony Butler, 35, has also been charged in connection with the slaying. Police said that a fourth and still unnamed person, possibly a woman, may have been involved as well.

Police and FBI agents have been forcusing their investigation on a small cadre of militant Islamic fundamentalists identified as the Islamic Guerrillas of America (IGA), who strongly support the policies of Khomeini Salahuddin is a leader in the organization.

The IGA has been the subject of an eight-month D.C. police investigation that began after the November seizure of the American hostages in Tehran.

Sources said police had taken hundreds of photographs of individuals at marches, Muslim gathering places, beside taxicabs and in front of vending stands on the street. Most of those photographed are thought to be possible members of the IGA. It was from these photographs that Salahuddin was identified by two people as the man who allegedly killed Tabatabai.

At least one high-ranking police official privately questioned the propriety of the investigation. "I am not going to talk about intelligence-gathering when we're not supposed to be doing it," the official said.

Under D.C. police department guidelines adopted in August 1976, intelligence-gathering is limited to criminal activities. Surveillance of noncriminal activities is prohibited.

The guidelines were adopted after the City Council complained in 1975 about reports that undercover officers collected unnecessary personal data on some antiwar and civil rights activists -- including some council members -- in the late '60s and early '70s.

Police officials said they have evidence to show that some members of the IGA have been responsible for threats against and assaults on persons at the Islamic Center on Massachusetts Avenue NW. Two alleged IGA members were arrested in April and charged with assaulting Abdel Osman, assistant director fo the Islamic Center.

Police also said the IGA has distributed pamphlets urging their followers to fight the enemy with guns, daggers and knives.

Two days after the Tabatabai assassination, a confidential memorandum was sent to Assistant D.C. Ppolice Chief William Dixon, who oversees the department's investigative services division. That memorandum stated that the Tabatabai assassination shows that the investigators had not been "crying wolf" during their months of surveillance.

Police said that their investigation indicated that the IGA was headquartered in the Iranian Interests Section, where Sebrahim Sadjadi is the Iranian government's only diplomatic reresentative in the United States. t

Police contend that Bahram Nahidian, a Georgetown rug merchant, key pro-Khomeini spokesman and Sadjadi's brother-in-law, is the leader of the IGA.

The group itself consists of about 30 to 50 "hard-core" members, police said, about half of whom are natives of Islamic countries in the Middle East while most of the rest are black Americans.

Police said they have identified among the group's members Marshall Fields, who was arrested on Christmas Day 1974, after he rammed his car into the White House gate and then held police at bay for four hours as he stood less than 20 feet from the White House doors, with what he claimed were explosives strapped to his body.

IGA activities consist largely of ditributing pamphlets, police said. But the group has reportedly discussed planting bombs in a Metro station and shooting down a plane. Neither of those actions ever took place, however.

Police said Nahidian and Salahuddin as well as some other IGA members recently traveled to the middle East and that some of the group have undergone terrorist training there.

As a result of the investigation, D.C. police recommended two months ago that Nahidian be charged with inciting a riot as the result of IGA activities at the Islamic Center and other groups activities, police said.

However, sources said, that recommendation was denied because federal prosecutors believed the evidence was too circumstantial.

Nevertheless, because of the D.C. police investigation, Montgomery County police were able to show pictures of IGA members to witnesses within hours of the assassination.

Yesterday, meanwhile, the FBI began questioning four of five American citizens who work at the Iranian Interests Section about Salahuddin, who had worked there after the expulsion of the Iranian Embassy staff in April, police said.

Initially, teh Algerians, who are nominally responsible for overseeing Iranian government interests in the United States in the absence of a formal Iranian presence here, had balked at allowing the FBI to question the employes.

But a compromise was worked out, allowing the FBI to interview them at a neutral cite in the presence of attorneys representing the Algerians.

The FBI's presence at the interests section of 2139 Wisconsin Ave. NW, had been precipitated by a tip that Salahuddin had taken refuge there. The FBI quickly staked out the area and accompanied by a State Department official, requested permission to search the offices.

However, Algerian Minister-Counselor Sims Tahar Debagha refused. A potential diplomatic incident was avoided when Debagha offered to search the Iranian Interests Section himself.

"He personally inspected the interest section and gave me personal assurances that he (Salahuddin) was not there," said Carleton Coon, who is director of the Algerian, Libyan, Moroccan and Tunisian desk of the State Department. "I conveyed them to the FBI."

Coon called the Algerian refusal to admit the FBI a matter of principle.

"If they'd invited the FBI in, they'd be giving away a principle. They've given every evidence of sharing a desire that Salahuddin be apprehended," he said. "We don't want to put ourselves in the position of the Iranians, who knocked off our embassy in violation of international law."

The FBI said it had alerted police officials in Canada and several Caribbean countries to be on the lookout for Salahuddin. FBI officials said they had information that indicated the IGA maintains contacts with dissident groups in those countries.

As part of their investigation Friday, the FBI spent five hours poring over the records on an indoor shooting range in Annandale in an effort to determine whether any IGA members had practiced firing pistols there during the last year.

Martin Joyce, the owner, said the FBI showed him about 35 photographs of people, none of whom he said he recognized. Four of them were Iranian women, he said, the rest clearly Middle Eastern or American. At the same time, the agents, he said, examined 5,000 signed slips filled out by those who purchased ammunition there.

At the Bethesda home of Tabatabai yesterday, headquarters of his Iran Freedom Foundation, more than 40 anti-Khomeini Iranian expatriates were preparing for today's march.

Plans now call for the march to begin at the west steps of the Capitol at 1 p.m. From there the march will proceed down Pennsylvania Avenue, turn left and follow Constitution Avenue, turn right on 15th Street NW, ending at Lafayette Square. The march and rally is scheduled to end at 5 p.m., the organizers said.

The organizers said that the foundation and its members have received more than 1,000 threats from what they called "Khomeini mobs," all of them prompted by today's march. They also claimed that pro-Khomeini partisans have been calling Iranians to tell them that the march has been canceled, a rumor they vehemently denied.

The organizers hailed Tabatabai as a martyr.

"His blood brought us together," one organizer said. "We have put our differences aside for one purpose -- to return home."

Three other demonstrations by Iranians are scheduled for today, police said. Altogether, as many as 10,000 people may participate.

The Moslem Student Association, which is pro-Khomeini, will hold a rally in Lafayette Square from 9 p.m. to 11 a.m. and then march up Connecticut Avenue to Dupont Circle. Another rally there is scheduled to last until 7 p.m., police said.

The Iranian Students Association, a Marxist anti-shah group, will assemble in the Ellipse from 10 a.m. to 2 p.m., then march in a circle around the Iran Freedom Foundation-Tabatabai rally in Lafayette Square, then back to the Ellipse.

The Confederation of Iranian Students, another Marxist, anti-shah association, will rally in the southwest quadrant of the Ellipse from 1 p.m. to 5 p.m.

The four nearly simultaneous marches are worrying police, who say they fear that the Moslem students will refuse to move from Lafayette Square in an effort to confront the Tabatabai-Iran Freedom Foundation marchers.

Police also said there will be a chance that the Iranian Students Association might clash with the Tabatabai group when it passes up 15th Street near the Ellipse.