The Internet passed a major milestone last week … probably.
According to estimates by Internet Live Stats — and trumpeted by Internet pioneer Tim Berners Lee — the World Wide Web now officially hosts a billion domain names, or roughly one Web site for every three Internet-users on the planet. It’s a profound feat for a technology that’s only been around 20-odd years. And it’s a pretty bold statement on Internet Live Stats’s part, as well: As it turns out, it’s almost impossible to put a number on the ever-shifting Internet.
Since August 1995, a British Internet company called Netcraft has tried. It basically uses a series of bots to ping every Web site, evaluate whether they’re active, and pull the data into a monthly report. (In August, for instance, Netcraft estimated that 992 million Web sites were online; of those, domains ending in .xyz saw the greatest growth.) Internet Live Stats then runs formulas against those monthly reports that, together with historical data and current projections, informs the site’s “live” counter.
According to that counter, the Internet passed a billion pages on Sept. 17.
But even Peter Collins, Internet Live Stats’s overseer, admits that’s an inexact science — more an educated estimate than an actual “statistic,” with all the open-and-shut authority the word implies. Netcraft’s surveys are not 100-percent conclusive: sending too many requests from bots could overload a server, so Netcraft sometimes just takes a sample and extrapolates from there. Internet Live Stats, meanwhile, has long recorded unpredictable fluctuations in the number of “inactive” sites, which means the Internet could easily be down below 1 billion pages next month. Collins also said the site is missing some data that he said could make its formulas more accurate: He wishes Google would provide better, more timely search data, for instance.
Still, the estimate and this particular milestone serve an important role, giving shape to an otherwise boundless, ill-defined Internet. Thanks to estimates from Netcraft, we know that 80 percent of all registered domains are just “parked,” or undeveloped: There’s no actual, unique content on that site. We know that the greatest jump in Web site volume occurred between 2011 and 2012. And over time, Collins thinks, we could know exactly how and when the modern Web developed, and how that’s come to shape our offline world.
“We strongly believe in the value of keeping track of the history of the Internet,” Collins said. “This is something very important because it gives a sense of the development and the usage through its brief history … and ultimately allows us to [understand] the resulting changes in our culture and society.”