Government Increasingly Turning to Data Mining
Thursday, June 15, 2006
The Pentagon pays a private company to compile data on teenagers it can recruit to the military. The Homeland Security Department buys consumer information to help screen people at borders and detect immigration fraud.
As federal agencies delve into the vast commercial market for consumer information, such as buying habits and financial records, they are tapping into data that would be difficult for the government to accumulate but that has become a booming business for private companies.
Industry executives, analysts and watchdog groups say the federal government has significantly increased what it spends to buy personal data from the private sector, along with the software to make sense of it, since the Sept. 11, 2001, attacks. They expect the sums to keep rising far into the future.
Privacy advocates say the practice exposes ordinary people to ever more scrutiny by authorities while skirting legal protections designed to limit the government's collection and use of personal data.
Critics acknowledge that such data can be vital to law enforcement or intelligence investigations of specific targets but question the usefulness of "data-mining" software that combs huge amounts of information in the hopes of finding links and patterns that might pick someone out as suspicious.
Dialing for Data
Recent reports about the National Security Agency's effort to acquire phone call records highlights the government's growing interest in the technique.
"The only question we would have is at what rate would the demand be increasing," Wayne Johnson, a financial analyst at Raymond James Financial Inc., said of the government's interest in buying commercial data and related software.
It is difficult to pinpoint the number of such contracts because many of them are classified, experts said. At the federal level, 52 government agencies had launched, or planned to begin, at least 199 data-mining projects as far back as 2004, according to a Government Accountability Office study. Most of the programs are used to improve services, such as detecting Medicare fraud and improving customer relations. But a growing number of agencies are exploring the technology to analyze intelligence and assist in the hunt for terrorists.
Another GAO report released in April found that of $30 million spent by four government agencies last year on services from data-crunching companies, 91 percent was for law enforcement or counterterrorism.
The hope is that the technology can help to discern and thwart threats just as businesses have used it for years to predict consumer behavior on buying cosmetics or repaying mortgages, for example.
Companies keep an increasing amount of data about everyone -- tracking their buying, travel, bank transactions and bill-paying habits. Data mining uses mathematical formulas to look for patterns in those behaviors. The results could enable the grocery store to send out targeted coupons, or, in theory, help the government decide how likely it may be that someone is linked to terrorist groups.
The Education Department's Project Strikeback uses mining methods to compare its databases with the FBI and verify identities. The Defense Department's Verity K2 Enterprise program searches data from the intelligence community and Internet searches to identify foreign terrorists or U.S. citizens connected to terrorists. A Navy program analyzes data to try to predict where it might find small weapons of mass destruction and narcotics smuggling in the shipping industry.