Why wait for the government to release official numbers on auto sales, home sales and retail sales when the trends could be gleaned weeks or even months earlier by analyzing publicly available data online?
Five years ago, only 2 percent of investment firms were incorporating Twitter analysis and other forms of “unstructured” data into their trading decisions, according to a report by Adam Honore, a research director at Aite, a financial services consulting group based in Boston. By 2010, the share of companies experimenting with this technology jumped to 35 percent. Today, Honore said, that number is closer to 50 percent.
“Big data is fundamentally changing how we trade,” Honore said.
‘Data in motion’
Richard Tibbetts, chief technology officer at StreamBase, a Lexington, Mass., company that provides tools for analyzing large amounts of data, calls it “examining data in motion.” The trick is to be able to find the digital smoke signals amid all the other stuff. Traders who were analyzing Twitter for unusual activity, for instance, were able to get the news of Osama bin Laden’s death and a massacre in Norway hours before the information was officially confirmed, giving them a significant jump on their colleagues who learned of the events through traditional news sources.
“The new generation of trader expects to have dozens of tools at their fingertips instead of just a Bloomberg terminal,” Tibbetts said.
Hawtin began experimenting with trading on a social-media sentiment algorithm in the spring of 2011, tapping $40 million from his now-closed hedge fund. He has repeatedly warned potential investors that there is a high level of risk. “It’s a very new area we don’t fully understand yet,” he said. But the interest in his project was so great that in April he began offering his technology to retail investors.
In addition to its efforts to gauge the collective mood of the world, the company now examines messages on Twitter, Facebook and other social-media outlets to create measures for individual stocks and commodities.
On a recent weekday, Hawtin was studying his global sentiment monitor when he noticed something troubling, a surge in anxiety after two days of relative calm.
After deliberating for a few minutes, he decided it was too early to take any action. If the anxiety continued to trend up the following day, he said, he would probably start selling.
“There’s a delay between how you’re feeling about your economic situation and having that sentiment turned into a decision like buying or selling a stock or adjusting your portfolio,” he said.