Documents three Americans brought back from the "Crimes of America" conference in Tehran were photocopied by U.S. Customs agents as part of an investigation of their violation of the presidential travel ban to Iran.

The three men, Rev. Paul M. Washington of Philadelphia, Lennox S. Hinds, a Rutgers University law professor, and Leonard Weinglass, a Los Angeles attorney and civil rights activist, complained about the government action when they returned to the United States Friday night. A Customs supervisor said the documents were returned after copies were made. a

Richard Davis, assistant secretary of the Treasury for enforcement, said yesterday that Customs has authority to seize materials brought into the country illegally or for use as evidence in an investigation.

Attorney General Benjamin R. Civiletti announced yesterday that Customs was making the investigation at his request. The results would be reviewed by both the criminal and civil divisions of the Justice Department, he said.

Ten Americans, including former attorney general Ramsey Clark, attended the conference. They were warned before leaving that they would be investigated for violating the embargo on trade with Iran.

In Tehran, meanwhile, five other members of the unofficial American delegation delivered a box of material to the militants at the U.S. Embassy but were denied a meeting with the hostages, held since last Nov. 4.

Clark has been criticized in both the United States and Iran for making the trip. Some members of Congress have called for his prosecution, and Tehran Radio called him the "vilest American agent."

Sen Robert Byrd (D-W.Va.), the Senate majority leader, said yesterday that "we've been giving too much notice to this aberration."

A Customs Service spokesman said the returning Americans would not be detained or have their passports confiscated.

Violations of the embargo on Iran are punishable by fines up to $10,000 and 10 years in prison for willful criminal violations.

The applicable law, the International Emergency Economic Powers Act, was enacted in late 1977. A Justice Department spokesman said yesterday he knows of no prosecutions yet. But he noted it would have been used if necessary to stop American athletes from participating in the Olympic games in Moscow this summer.