By MICHAEL J. SNIFFEN
The Associated Press
Wednesday, September 5, 2007; 9:26 PM
WASHINGTON -- The Homeland Security Department scrapped an ambitious anti-terrorism data-mining tool after investigators found it was tested with information about real people without required privacy safeguards.
The department has spent $42 million since 2003 developing the software tool known as ADVISE, the Analysis, Dissemination, Visualization, Insight and Semantic Enhancement program, at the Lawrence Livermore and Pacific Northwest national laboratories. It was intended for wide use by DHS components, including immigration, customs, border protection, biological defense and its intelligence office.
Pilot tests of the program were quietly suspended in March after Congress' Government Accountability Office warned that "the ADVISE tool could misidentify or erroneously associate an individual with undesirable activity such as fraud, crime or terrorism."
Since then, Homeland Security's inspector general and the DHS privacy office discovered that tests used live data about real people rather than made-up data for one to two years without meeting privacy requirements. The inspector general also said ADVISE was poorly planned, time-consuming for analysts to use and lacked adequate justifications.
DHS spokesman Russ Knocke told The Associated Press on Wednesday the project was being dropped.
"ADVISE is not expected to be restarted," Knocke said. DHS' Science and Technology directorate "determined that new commercial products now offer similar functionality while costing significantly less to maintain than ADVISE."
Earlier, DHS said testing would resume once appropriate privacy analyses and public notices were completed.
ADVISE was one of the broadest of 12 data-mining projects in the agency.
A DHS research official said in 2004 it would be able to ingest 1 billion pieces per hour of structured information, such as databases of cargo shippers, and 1 million pieces per hour from unstructured text, such as government intelligence reports.
The system was supposed to identify links between bits of information that could otherwise go unnoticed. And it would graphically display results in charts of relationships and links.
A DHS workshop report in 2004 said it hoped to answer queries like: "Identify any suspicious group of individuals that passed through customs at JFK (airport in New York) in January 2004."
The GAO said in March that DHS should notify the public about how an individual's personal information would be verified, used and protected before ADVISE was implemented on live data.
Then, in separate reports released without fanfare in July and August, the DHS inspector general and privacy office concluded that between 2004 and 2007, three pilot tests of ADVISE used personally identifiable information without first issuing required privacy impact assessments. The privacy office said this "created unnecessary privacy risks."
The errors were in pilot programs involving weapons of mass effect, immigration enforcement and the DHS intelligence analysis office, the privacy office said.
This is the second such error at DHS.
The Secure Flight program to screen domestic air travelers was blocked by Congress after it acquired live personal data for testing. That program has since issued a privacy impact assessment, dropped use of commercial data such as personal credit card histories, and will begin tests this fall.
"Data analysis programs hold real promise for protecting America, but they need to be tested using dummy data before being used on real data," said James Dempsey, policy director for the Center for Democracy and Technology, a civil liberties group. "Why this mistake keeps getting made over and over again, I don't understand."
Senate Judiciary Committee chairman Patrick Leahy, D-Vt., said database tools "must be closely supervised to prevent abusing the rights and the privacy of ordinary Americans. All too often, the Bush administration has treated safeguards for databases as a disposable afterthought if at all."
Sen. Joseph Lieberman, I-Conn., chairman of the Senate Homeland Security Committee, said DHS "must follow federal privacy laws _ in this case the E-Government Act which requires privacy impact assessments before personal information can be used _ in order to maintain public support for these new technologies."
Among the data the privacy office found had been plugged into ADVISE pilot projects were:
_The no-fly list of people barred from domestic air travel and the list of people who require special inspections before flying.
_More than 3.6 million shipping records from a commercial data provider with names of cargo shippers and consignees.
_Terrorist Screening Center lists of people who tried to cross the U.S.-Canadian border at a port-of-entry.
_Classified intelligence reports about illicit traffic in weapons of mass effect.
_Lists of foreign exchange students, immigrants under investigation and people from special interest countries.
Although Knocke said ADVISE "was never used in an operational environment" and DHS had assured Congress in 2006 it was not operational, the inspector general found that "on at least one occasion, the data was used to produce classified intelligence information."
At Lawrence Livermore, analysts used the pilot system "to uncover previously unknown connections between organized crime and terrorism," the IG found in a July report quietly made public last month.
The privacy office concluded that although required privacy analyses were ignored, the Privacy Act was not technically violated because the live data were covered by privacy notices issued earlier for other programs that originally gathered the information. Dempsey argued those were too vague to alert citizens how ADVISE would use their data.
DHS inspector general report:
DHS privacy office report: