A government counterterrorism initiative that collects and stores information about alleged suspicious behavior has been challenged by civil liberties groups, which filed a lawsuit Thursday claiming the program leads to the investigation of people involved in no criminal activity.

The lawsuit, filed in San Francisco by the American Civil Liberties Union and Asian Americans Advancing Justice-Asian Law Caucus, was brought on behalf of five men who said they were unfairly targeted by the government’s Suspicious Activity Reporting program, created several years ago to share information that could detect, prevent or deter a terrorist attack.

The lawsuit says plaintiffs came under government scrutiny for taking photographs, trying to buy computers or standing in a train station.

“This domestic surveillance program wrongly targets First Amendment-protected activities, encourages racial and religious profiling and violates federal law,” said Linda Lye, staff attorney with the ACLU of Northern California.

One of the plaintiffs, James Prigoff, is an 86-year-old photographer who has lectured at universities and has had his work exhibited at the Smithsonian and other museums. The FBI went to his home in Sacramento and questioned a neighbor about him after he photographed a piece of public art in Boston called the Rainbow Swash, which is painted on a natural-gas storage tank.

“All I was doing was taking pictures in a public place, and now I’m apparently in a government terrorism database for decades,” Prigoff said. “This is supposed to be a free country, where the government isn’t supposed to be tracking you if you’re not doing anything wrong.”

The FBI visited Prigoff several months after security guards at the Rainbow Swash told him to stop taking photographs.

The Suspicious Activity Reporting initiative, operated by the Justice Department, the Department of Homeland Security and state and local law enforcement, stores information about tens of thousands of citizens and legal U.S. residents who have not been accused of a crime but have appeared, in the eyes of a police officer, neighbor or security guard, to act suspiciously.

A 1978 Justice Department regulation prohibits the collection of “criminal intelligence information” unless there is reasonable suspicion of criminal activity. But the ACLU said that the Justice Department’s standard for the SAR program requires only behavior that “may be indicative” of terrorism planning “or illicit intention.”

The Justice Department did not immediately respond to a request for comment.

Another plaintiff in the lawsuit, Wiley Gill, was identified two years ago as a “Suspicious Male Subject in Possession of Flight Simulator Game,” according to a 2012 SAR report that was obtained by the ACLU of California through a Public Records Act request. He was identified as “worthy of note” because he converted to Islam and has a “pious demeanor,” according to the information the ACLU obtained.

“The only reason that someone deemed Mr. Gill ‘suspicious’ is because he is a devout Muslim, not because he has done anything wrong,” said Nasrina Bargzie, an attorney at Asian Americans Advancing Justice-Asian Law Caucus.