By Julie Wan
Sunday, November 7, 2010; W24
Tameka Clanton sits before a 61-inch screen wearing a pair of 3-D eyeglasses. As if ready to do battle in a video game, she clicks the avocado-size hand controllers and sweeps her arms back and forth. The X-ray-like image on the screen expands, shrinks, then rotates, following her hand movements.
But Clanton is not trying to rack up points. More serious matters are at stake. A biomechanics researcher at the University of Maryland Medical Center in Baltimore, Clanton is demonstrating new software that uses the virtual reality technology employed by video game developers to view human anatomy. Called iMedic, the program creates a 3-D X-ray that allows doctors to examine the body from every angle.
This software exemplifies how the video gaming industry has penetrated academia, offering researchers and students a new way to understand techniques that aren't always easy to teach. In pockets around the country, some video game companies are veering away from the entertainment industry to focus solely on creating what they call "serious games." One company, Breakaway Games in Hunt Valley, an East Coast gaming development hub just outside Baltimore, has switched completely in the past three years to developing only games for training. Its clients include the medical schools at the University of Maryland and Johns Hopkins University.
Over the past decade, an entire movement has formed around applying state-of-the-art computer games to new sectors. In 2002, the Woodrow Wilson Center for International Scholars in Washington and Digitalmill in Portland, Maine, created the Serious Games Initiative, which has led the quest to develop computer and video games to help solve world problems in education, health care, public policy, corporate management and national defense.
"The sentiment to use games in learning has always existed," said Ben Sawyer, president of Digitalmill and co-founder of the Serious Games Initiative and one of its offshoots, the Games for Health Project. Flight simulators, computerized war games and practice space stations have been used for decades to train pilots, soldiers and astronauts. "What's changing drastically now is the capability to inject much more robust and usable applications in the form of video games and computer simulations into these environments like never before."
Nowhere is this more difficult than in the medical field, where doctors are making critical decisions and responding to unpredictable reactions from the human body. The interplay between these factors has presented serious game developers one of their greatest challenges yet. But as the technology becomes more refined, it is gradually earning the trust of medical professionals.
For example, the University of Maryland Medical Center has transformed an entire wing of its seventh floor into the Maryland Advanced Simulation, Training, Research and Innovation (MASTRI) Center, which officially opened at the end of 2006 and has been growing ever since. Every Tuesday, surgical residents meet for training in once-functional operating rooms converted into simulation labs. The trainees are presented an emergency scenario -- say, a "patient" with a blocked airway -- and must respond using either a virtual reality computer program or a hybrid that includes a mannequin simulator.
The residents must identify a course of action, and staff members remotely control whether the "patient" will improve or decline based on the residents' decisions. When mistakes are made, the panic for trainees sometimes lingers long after the simulation is over. "It's purposely stressful," said Ivan George, director of advanced technologies and special projects in general surgery. "One time, it got so real one of the residents wouldn't talk to me for 20 minutes after the procedure."
Afterward, the residents watch a video of the session, much like football players reviewing plays after a game.
Beyond emergency training scenarios, though, the use of gaming and simulation technology in medicine has been slow to gain footing, mainly because the technology still lacks the ultra-high-fidelity realistic simulations that can accurately reproduce what doctors encounter in the field. "The most complex human-made machine, such as a space shuttle and the simulator that comes with it, is ultimately less complex to model than a single organ, like a pancreas," said Adrian E. Park, chair of the MASTRI Center and head of general surgery at the University of Maryland Medical Center.
"We are in the earliest days of trying to simulate all the possible biological possibilities in one single organ," he said. "We can do things to simulate fairly basic steps. But if you want the full reaction of a pancreas or liver as it functions in health and disease, boy, we are just a long, long way from that."
In surgery, simulation training is limited to a small number of specific tasks, such as removing a gall bladder, which is practiced on a laparoscopic virtual reality machine. The machine, a sort of high-end computer console, mimics a surgical procedure in which cameras at the end of long instruments are inserted inside the body, while surgeons watch on a nearby screen. The graphics on the screen portray a fairly realistic image, and the surgical scopes that the students manipulate can prod, snip and even feel resistance, just as in an actual surgery.
If something goes wrong, such as a cut in the wrong place, the program assigns a low score. But getting a high score on a virtual reality laparoscopic surgery doesn't ensure that a resident can successfully remove a real gall bladder. "When you fly a flight simulator, you can earn one hour of real time in the air from one hour spent in a flight simulator," George said. "In the clinical world, that doesn't exist."
Medical training is still based primarily on an apprenticeship model, and surgery, in particular, can be a mysterious process to master, combining factual knowledge, cognitive decision-making and psychomotor skills. If medical schools can replicate how these disparate pieces come together in the mind and hands of a master surgeon, they can help their students better emulate the experts, researchers say.
At Johns Hopkins, Gregory Hager, a computer science professor who researches medical devices, is using the da Vinci Surgical System -- the only robot approved by the FDA to help perform minimally invasive surgery -- to deconstruct surgical procedures into measurable components and imitable parts. The robot is able to quantify performance by recording every motion surgical residents execute and then comparing the data to the gestures of expert surgeons. From this, residents can get specific feedback on how to alter their movements and techniques.
At the University of Maryland Medical Center, Gyusung Lee, assistant professor of surgery, oversees a research project with similar goals but different means. In the lab where iMedic is run, Lee employs motion-capture technology -- used to create "Madden Football" and "The Lord of the Rings"-- to study the movements of experienced surgeons. A platform that functions like a large Wii Fit board captures a surgeon's every move. A vest and motion sensors are attached to the surgeon's body and arms. Video game developers use this method to record the movements of football and basketball players in creating a library of motions from which gamers can choose. This is the only surgical lab in the country using cameras with real-time capture -- with accuracy down to the submillimeter -- for medical research.
Lee's project studies how some surgical procedures take a toll on surgeons, many of whom develop back or neck pain or carpal tunnel syndrome from straining to view the monitors while holding large instruments in awkward positions for hours. His aim is to share optimal postures and techniques practiced by master surgeons, so that residents can adopt better habits and stronger skills that will allow them to perform without injuring themselves.
Although there is skepticism about whether video gaming and simulation should be taken seriously in medical training, the work and investment that have gone into the technology suggest that it will have a broad future. Medical professionals and video game developers are starting to work together more to reach solutions. Gerald Moses, director of the MASTRI Center, also leads a grass-roots movement called Advanced Initiatives in Medical Simulation, which has been lobbying on Capitol Hill for about eight years to raise support for simulation technology nationally.
"I fully believe these efforts and technologies will converge somewhere, whether sooner or later," said George, the advanced technologies director at the University of Maryland Medical Center. "Maybe I've just watched too much 'Star Trek,' but I believe this cross-pollination of different disciplines is how we're going to reach solutions. And gamers are one part of it."
Julie Wan is a freelance writer in Washington. She can be reached at firstname.lastname@example.org.