"How do you authenticate what you're looking at? How do you know this isn't some kind of fly-by-night operation that's put up this Web site?" asks librarian Patricia Wand of American University.
Students typically search only the most obvious parts of the Web, and rarely venture into what is sometimes called the "Dark Web," the walled gardens of information accessible only through specific databases, such as Lexis-Nexis or the Oxford English Dictionary. And most old books remain undigitized. The Library of Congress has about 19 million books with unique call numbers, plus another 9 million or so in unusual formats, but most have not made it onto the Web. That may change, but for the moment, a tremendous amount of human wisdom is invisible to researchers who just use the Internet.
When Google first appeared, in 1998, it seemed like a throwback. Rather than a jazzy "portal," it was just a plain ol' boring search engine.
"For a lot of kids today, the world started in 1996," says librarian and author Gary Price.
And yet Berkeley professor Peter Lyman points out that traditional sources of information, such as textbooks, are heavily filtered by committees, and are full of "compromised information." He's not so sure that the robotic Web crawlers give results any worse than those from more traditional sources.
"There's been a culture war between librarians and computer scientists," Lyman says.
And the war is over, he adds.
In the early days of search engines, finding information was like fishing in a canal: You might hook something good, but you were just as likely to reel in an old tin can or a rubber boot. Now you often find exactly what you want.
One reason Google works so well today is that there's so much for its robotic crawlers to explore. Google initially searched about 20 million Web pages; the company's home page now boasts that it searches 3,307,998,701 pages.
"In 1996, if you tried to Google someone, if Google existed, it wouldn't have been a very satisfying experience," says Seth Godin, author of a number of best-selling e-books. "We hit a critical mass of really valuable stuff that was online, I think, about 2000."
The expansion of the information universe makes the navigational tool all the more valuable. And yet the search function at first seemed to be an unglamorous computer application. The pioneering search engine companies, including Yahoo!, Excite, AltaVista and Lycos, wanted to transform themselves into something snazzier, a "portal," the full gee-whiz Internet Century home page that would offer the user a link to everything between here and Neptune, plus plane tickets.
But the history of computer technology is full of companies that failed to see the potential glory right in front of them. In the early 1980s, IBM thought that the "operating system" within the computer wasn't nearly as important as the hardware, the box itself. And then Microsoft, which benefited from that oversight, became so focused on software programs that it was slow to capitalize on the Internet revolution, leaving Netscape to create the first commercial Web browser. And then almost everyone underestimated Search.