The pace of data creation on the Internet is growing at an exponential rate. In any 48-hour period in 2010, more data was created than had been created by all of humanity in the past 30,000 years, according to a presentation by entrepreneur Yuri Milner. By the year 2020, that same amount of data will be created in a single hour. “Information wants to be free,” declared writer Stewart Brand, and so does all of this data. In fact, the big data movement — organizations scrambling to make sense of all this data — may be the single most profound trend that the Web has created in the past 12 months. It has also given rise to a group of Digital Libertarians who see the beginning of a new era in which total data transparency makes it possible to regulate the activity of corporations, governments and individuals.
Data will set you free.
Digital Libertarianism seems like a radical concept until you consider the fact that - across many industries - regulators are no longer able to keep up with all the data that is being created every day. The most obvious example is the financial markets, where even traders and investors talk about the creation of a vast new "shadow market" where financial instruments are just too complex to regulate and algorithmic traders, using powerful supercomputers to analyze all of the data, can yank the market down in a matter of seconds. Viewed in this way, is it actually possible that it would be in a financial institution’s competitive best-interest to release as much data as possible into the marketplace, in exchange for freedom from regulatory shackles?
How can this be the case? Isn’t proprietary data the key to competitive success in the markets? Not always — consider all of the open innovation initiatives formed to solve the world’s most difficult problems in fields like biology and chemistry and physics. Large corporations have found that full data disclosure, offered in good faith and made available to researchers, can be a way to accelerate growth — especially if the crowd gets involved. It can also be a brake on business practices that harm society. Take taxes, for example. Full data disclosure can also solve for problems like the propensity of individuals and corporations to play fast and loose with their taxes. Governments have always struggled to collect the full tax burden from individuals and corporations.
Consider your own experience on a site like Facebook. The more information and data that you make available to others, the more that you are held accountable to others. On the flip side, all this data leads to new types of experiences, like the ability for corporations to perfectly customize their products to your tastes and needs. Now that Facebook has created the Timeline, people will know exactly what you read, where you were last night, what music you listened to and what Web site you think has the best “LOL kittens.” Your personal life just became public. Imagine if corporations had to disclose as much data about everything they did.
As Supreme Court Justice Louis Brandeis once said: “Sunlight is the best disinfectant“? Perhaps that could be the case here as well.
The idea of making data completely open and transparent is gaining traction in tech circles. Consider New York City's recent overture to the tech crowd, opening up all of its data about the city to developers and entrepreneurs. Government officials could easily hide information about crime reports, public health issues, building complaints and traffic reports. But what if this information could become the stimulus for new innovation by the crowd to solve problems that the government could not?
An increase in the amount of data made freely available to all could lead to governments and businesses doing more good for society. That may sound hopelessly idealistic, but so was the original premise of the Web. As participants in a democracy, we have rights as well as responsibilities. In an age of Wikileaks and forced data transparency, one new responsibility might be to take more than a passing interest in all of the data being created every 48 hours, and instead, use it to hold our society’s institutions fully accountable.
Read more news and ideas on Innovations: