Fast Forward: 'Augmented reality' fuses your world and the Web

By Rob Pegoraro
Sunday, November 22, 2009

The cameras on some new phones don't show the world as you've known it.

Instead of just viewing the usual landscape of people, places and things on their screens, you see circles, rectangles and icons floating on top of the scenery. Tap one to display a snippet of Internet data about whatever lies behind that tag. As you look around, the view on the phone's display shifts accordingly, presenting new shortcuts to whatever the Web knows about your surroundings.

The concept goes by the name augmented reality, and it's been quietly bringing one of the Internet's hokiest promises to a mainstream audience.

Remember all the hype about virtual reality, in which we'd don headsets to immerse ourselves in some version of the Star Trek holodeck? Augmented reality turns this from a science-fiction idea into something you can experience just by holding a smartphone in front of you at eye level -- no goofy goggles or helmets needed.

For that to happen, though, mobile phones had to acquire a few prerequisite capabilities: a fast Internet connection, a high-resolution screen, Global Positioning System reception and a compass. In other words, first the phone had to be able to look up things on the Internet, then it had to be able to show them to you, then it had to find itself on a map, then it had to orient itself in 3-D space.

As a result, "AR" programs didn't begin to appear on consumer hardware until last year, and many otherwise brainy smartphones still cannot run them -- for example, the original iPhone and iPhone 3G and Palm's Pre and Pixi devices lack compasses.

The first such program I tried was an application for Google Android phones called Wikitude ( This free program, since released for the iPhone as well, lets you browse the Wikipedia encyclopedia and the Wikitravel tourism guide, among other Web resources.

I held a T-Mobile G1 phone as I stood in front of my house, slowly panned around the neighborhood and watched Wikitude point out the locations of not just nearby shops and restaurants, but old Civil War forts and other historical landmarks. The humble, printed travel guide seemed an endangered species.

Since then, Wikitude has picked up some company. Two popular iPhone programs, Yelp and UrbanSpoon, added augmented-reality interfaces for iPhone 3GS users this summer. Yelp's "Monocole" directs you to nearby restaurants bars and shops with rectangles that show each one's name, type, distance and one- to five-star rating from the site's users. UrbanSpoon's "Scope" overlays circles that display a restaurant's popularity; tap one to see details like the place's name, distance and price range (although its goofy sonar-ping sound effect, along with the cross hairs it puts on the screen, suggests that your next move should be to launch torpedoes rather than book a table).

Google's Android-only Sky Map application dispenses with an in-camera view to display the planets, stars and constellations in a given direction, regardless of obstacle -- point the phone towards the ground, and you can see where the Southern Cross would be if you could only see straight through the Earth.

But the most extensive, creative use of an augmented-reality interface comes in an Android and iPhone program called Layar (, which lets you superimpose data from (as of Thursday evening) 232 different sources. These layers include the usual restaurant, bar and movie databases, but also more intriguing material.

For example, you can look up Flickr photos taken around you, find nearby flu-shot locations, or see Twitter updates that have been tagged with geographic coordinates. Last month, the Sunlight Foundation, a government-transparency group, released a layer that maps out contracts granted under the American Reinvestment and Recovery Act.

Not all of these programs are tremendously helpful. When I fired up Yelp on an iPhone in the middle of downtown Vienna, it popped up so many links that I couldn't find a restaurant one block away. But the potential is almost mind-bending, especially if you extrapolate other things that smartphones could do with added processing power.

University of Maryland computer science professor Ben Shneiderman threw out some possibilities in an e-mail: recognizing paintings in a museum to display their provenance and artists' biographies, identifying flowers and trees during a nature hike, even looking up the names of people in your vicinity.

At that point, we could have a market for yet another visionary AR program: a database of streetlights, parking meters and telephone poles to warn distracted pedestrians when they're about to walk into one.

Living with technology, or trying to?

© 2009 The Washington Post Company