Agents of the U.S. Immigration and Naturalization Service, in a move one spokesman said could lead to the expulsion of several thousand Iranians from the United States, began yesterday to compile detailed files on all 50,000 Iranian students in this country.

The students, all of whom will be photographed, are being required to present their immigration papers, current addresses and a thorough accounting of their academic and financial standing at the schools where they are enrolled, a Justice Department spokesman said yesterday.

Immigration officials said that current plans do not call for any "round-ups" of Iranian students, but that, as the files are compiled, those students who have overstayed their visas, or who have violated the terms of their visas in other ways, will be fingerprinted, and deportaton proceedings will be initiated against them.

"There is no precedent for this kind of action in a peacetime situation," one administration official said yesterday, commenting on the scope of the operation.

President Carter ordered the action Saturday in an effort to head off demonstrations and violence that could endanger the lives of the more than 60 Americans being held hostage in the U.S. Embassy in Tehran.

Nationwide demonstrations Friday by Iranians, many claiming to be students supporting the embassy takeover, were marked by several incidents of violence at the hands of irate Americans. In Washington, Americans who turned out to heckle the marchers could be heard chanting Deportation! Deportation!"

Though most colleges and universities in the Washington area appeared yesterday to be complying with the INSdirectives, some administrators and attorneys expressed doubts about the legality of such measures.

Eric Heiberg, director of the international student office at Georgetown University, said he believed most of the 45 Iranian students there are opposed to Iran's policy and have legal visas. "We're looking into this as a possible question of harassment," he said.

Immigration lawyer David Carliner compared the tone, if not the scope, of the action to the internment of Japanese-Americans and Japanese nationals in this country during World War II.

"There seldom has been anything like this -- something of such a sweeping nature," Carliner said. "It's reaching people who are here legally and illegally. It's singling out one nationality because of a specific foreign policy concern. . . .

"There's a question whether the government is entitled to much of the information they're asking the universities to give. There are possible violations of the Buckley [privacy] amendment involved."

Immigration officials, who late yesterday afternoon were still drawing up detailed guidelines for the operation, said the only U.S. citizens and legal permanent residents are covered by the privacy laws. All information required of the Iranian students will have a direct bearing on their immigration status in the United States, in accordance with immigration law, the officials said.

Carter's new policy is a dramatic reversal of previous immigration directives concerning Iranians. In March, the administration effectively barred the deportation of Iranians for fear that their lives might be in jeopardy if they were forced to return to their politically unstable country.

These directives, which were to have remained in effect until at least June 1, 1980, have been revoked. Iranians whose visas have expired or who are otherwise subject to deportation will be sent home, like any other illegal alien, after appropriate legal proceedings.

Those who have reason to fear persecution in Iran may apply for political asylum in this country. But according to a Justice Department spokesman, "the burden of proof will be on the student. He's going to have to make a convincing case."

Immigration investigators in the District of Columbia, many of whom came to work yesterday despite the holiday, clearly were pleased with the new policy. "We'd like to be out at the demonstrations rounding them up," one said with an all-encompassing gesture.

But others expressed doubts about the short-term effectiveness of the new policy, which does not include such roundups and is aimed directly at students. p

"The ones who are actually in the schools are not really the ones we want to get," said one investigator. "And the ones we want are not likely to turn themselves in."

A spot check of compuses around the country yesterday indicated that college administrators do not expect wholesale deportaton of currently enrolled Iranian students.

"All of our Iranian students are here legally," said Cynthia Shephard Perry, director of international student affairs at Texas Southern University in Houston.

She said that the INS routinely is notified when foreign students drop out of school. "We have had some Iranians who aren't in school this fall coming by to ask if we're going to report them," she said. "We tell them yes."

Texas Southern, a predominantly black school, has more than 1,000 Iranians enrolled, according to Perry.

Bea Von Allmen, director of international student services at the University of Southern California, which also has a large Iranian student population (about 900) said some of the Iranians at the university in Los Angeles were worried about the image the anti-shah demonstrators presented to the American public.

"They were talking today about trying to organize something much more positive," she said. "They're naturally concerned about their welfare."

Von Allmen said she believed that only a small minority of the Iranian students at USC have taken part in the anti-American demonstrations. "But they're the noisy ones. They're the ones you hear," she said.

The Iranian student population in the United States has climbed dramatically in recent years, according to college officials and INS records.Many have come since the increase in world oil prices in 1973 made Iran a rich nation.

In 1973 about 15,000 Iranian students were in the United States. INS records show that 7,750 Iranian students arrived here in 1975, and 25,000 more came in 1977. INS records do not show how many left, though , some immigration officials admit they have no firm estimate on how many Iranians are here on student visas.