If the Internet is a library, the Web's search engines are the computer industry's desperate attempt to provide a reference desk. Unfortunately, the "library" of the Internet doubles in size every year, few of the shelves are actually parallel to the floor and every book in the stacks is liable to be redeposited on another shelf at random.
So even a good search engine is going to have problems finding what you're looking for. A bad search engine--that is, most of them--will do even worse. Consider the results I found while searching for the District of Columbia's home page (http://www.ci.washington.dc.us). I typed "washington, d.c." into the five most popular search sites for October, as ranked by researchers Media Metrix: Yahoo, Go.com, Lycos, Excite and AltaVista.
At Yahoo (http://www.yahoo.com), my query led me through two pages of site categories until I reached a lengthy, alphabetical list, in which the District's site appeared a third of the way down.
Go.com's Infoseek (http://www.go.com) yielded a fairly random list that began with a directory of Washingtonians' personal Web sites and included the city's site but also featured Microsoft's defunct Washington Sidewalk and the USDA Forest Service's home page.
Lycos (http://www.lycos.com) served up advertiser-paid links (to book hotel rooms or buy books), the weather forecast and a list of Lycos categories; the first search hit was for the National Academy Press. Others led to local chapters of the National Railway Historical Society and the University of Virginia alumni association, but the city's own site appeared nowhere in the first page of results.
Excite (http://www.excite.com) coughed up a nice thumbnail map of D.C. and categories in Excite's directory--D.C. radio, D.C. bars, D.C. restaurants and, for some reason, passports (remember to get your Ward 3 visa stamped!). Actual site links--to a list of personal Web sites, hotels, job sites and a few general Washington info resources, but not the city's page, followed.
Finally, AltaVista (http://www.altavista.com) offered a "RealName" shortcut to the District's site, but since this keyword link didn't list the actual address, I didn't know it was the real thing until clicking through. AltaVista also provided a few paid links and pointers to such oddball picks as Washington-themed chat areas at About.com and some K Street law firms.
I repeated this exercise with six other search sites: America Online (http://
www.aol.com), Ask Jeeves (http://www.ask.com), Google (http://www.
google.com), GoTo.com (http://www.goto.com), HotBot (http://www.hotbot.com) and Microsoft's MSN (http://www.msn.com). Google--a featured search engine at washingtonpost.com--did the best of the bunch; the others offered results that could only have been compiled by a computer. (Note to GoTo.com: The University of Washington is in Seattle, not D.C.)
Too many of these sites are too concerned about giving their sponsors good placement, too focused on keeping visitors around to look at ads and too dependent on inflexible search-software algorithms to be a match for any human reference librarian. But there are ways around search engines' blockheaded behavior.
The first thing to do is define your search. Are you looking for a particular site--MCI Center, the Philadelphia Inquirer, the city of Seattle--that you know exists? Or are you looking for information--bike trails, squash recipes, the world's tallest skyscrapers--without an idea of where it might be?
In the first case, start with the fastest approach, a Web browser's built-in search capabilities. The latest versions of Microsoft and Netscape's browsers send anything that looks like a search query--a phrase or something beginning with a question mark--directly to a search engine.
Of the two programs, Microsoft's Internet Explorer takes a smarter approach, displaying search results in a pane on the left of the browser window while using the right two-thirds of the window to display whatever site you select. Netscape, however, offers a better built-in search, returning both topical categories from its own directory and links to individual sites that have been reviewed by humans. It also offers a one-click option to pipe your query over to Google.
Google itself is an excellent option for "the site's on the tip of my tongue" searches. It ranks a site by how many other sites link to it--a popularity-contest approach that works surprisingly well. You can add a Google-search shortcut to your browser by visiting the Bookmarklets site (http://www.bookmarklets.
com/tools/search/srchbook.phtml) and bookmarking the "Search Google" link. When you select that from your bookmarks or favorites list, it will open up a small dialogue box asking what you want to search for; type your query and you'll go directly to Google's finds.
But if you want to find out information on a topic but have no idea what page might hold it, try Ask Jeeves and HotBot in addition to the likes of Google. Ask Jeeves touts its ability to accept questions in plain English ("Why are search engines so bad?"), but what's more useful is how it funnels your query to five other search engines, then displays the results on one page. HotBot, meanwhile, allows you to construct a highly specialized search, with pull-down menus to narrow your search by address, age, language, country, included multimedia and much more.
Finally, know when to bail out of the search engine entirely. When you find a site dealing in your topic of choice, see if it includes a page of links to related sites; if so, explore those links. In other words, find a site that can be your reference librarian. So, for instance, I'm better off looking for info about skiing in the area at the DCSki site (http://www.dcski.com) than flailing away in Yahoo. The sooner you can move from the computer intelligence of a search engine to the human intelligence behind a page of links, the more likely you are to get somewhere worthwhile.
Living with technology, or trying to? E-mail Rob Pegoraro at email@example.com.