When they’re assembling an onscreen production, Ken Lahti’s team works hard to get every detail just right. Script writers conduct hours of field research, collecting slice-of-life stories from regular people to spin into realistic works of fiction. Interactive designers base virtual characters’ actions off the movements of a person wearing a motion-capture suit. And they always hire professional voice actors to ensure that their characters are completely believable.
This team is not working in Hollywood, and you won’t be enjoying their animated tales at the local multiplex over a bag of popcorn. Rather, they are the creators of a new breed of online tests that you just might have to take the next time you apply for a job.
Digital pre-employment tests have grown more popular in recent years as innovations in technology and data interpretation have made them easier to administer and analyze. The tests can assess everything from personality traits to an applicant’s skills in mathematics, verbal reasoning and logic, and they are being used to zero in on strong candidates for a wide range of positions — stock clerks, restaurant supervisors, IT professionals and even senior executives.
At the heart of this trend is Arlington-based CEB, the company formerly known as Corporate Executive Board that is today one of the world’s largest talent assessment companies. Lahti is CEB’s vice president of product development and innovation, and his squad of industrial psychologists, data scientists and other professionals is working to transform the process of talent acquisition with a heavy dose of science.
For those who rely on traditional methods of résumé screening, Lahti’s research has led to some worrisome conclusions.
“There’s almost zero correlation,” between résumé accomplishments and job performance, Lahti said.
CEB believes its pre-employment tests offer a better approach for two key reasons: First, the tests are based on reams of long-term data that show what kinds of traits and skills correlate with success in a given company or industry. Second, CEB says the assessments help prevent bias from creeping into the hiring process, giving each candidate a fair shot to be judged solely on performance.
The assessments come in a variety of formats, but perhaps the most innovative are the elaborate multimedia simulations starring animated avatars. In a bank teller simulation, you might have to encourage a chatty character to move along when the line is backing up behind him.
CEB has invested in technology to make the movements and speech patterns of these onscreen characters as life-like as possible.
“It’s not ‘Avatar’ or ‘Titanic,’ but you spend a little,” Lahti said with a laugh.
The company has found that realistic avatars are preferable to live actors for the simulation scenes. In part, that’s because it keeps things easy for multinational companies. If the companies want to offer the same test to workers across the globe, the avatars can easily be altered to have the faces, outfits and languages that the worker is likely to encounter in a given culture.
Plus, with avatars, Lahti said, test takers “seem to be much more forgiving of details that are off or aging.”
The animations can take anywhere from several weeks to several months to complete. Animators start by creating characters, props and background environments for each scene. Then, voice actors and producers go to a studio to record vocals. They are joined by industrial psychologists, who listen in to make sure the actor is properly capturing the emotion and tone of the scene. Later, the animators incorporate motion-capture technology to assure that the movements are life-like.
The simulations sometimes incorporate “branching,” a format of assessment in which your answers to questions affect the onscreen situation. For example, if you’re asked to welcome an avatar as a new member of your team, the avatar will react based on your answers. If you aren’t very friendly, you might not get another chance to interact with the character in later scenes.
In addition to multimedia simulations, many CEB assessments also contain multiple-choice questions. Many of these, especially those that evaluate personality traits, do not have a true wrong answer. For example, a sample test for a bank teller asks the applicant to complete this thought: “The most accurate description of when I work best is ... ” Possible answers include “when the instructions are clear,” “when I am left alone” or “when I am under pressure.”
It is for this reason that Lahti believes the tests are difficult to game.
“You actually have to pick among equally plausible and equally socially desirable answers,” Lahti said.
Still, some of the questions contain possible answers that it seems candidates would be unlikely to select if they were trying to put their best foot forward. On the bank teller sample test, one question asks, “When we ask your most recent manager, how often will he or she say customers complimented your service?” One of the choices is “somewhat less often than others.”
But Lahti insists none of the choices are there simply to throw off test takers.
“If you have five response options and there’s one or more that no one picks, you might as well not even have that response,” Lahti said. “There actually is a non-trivial portion of the population choosing every single one of those.”
CEB can mix and match a variety of types of test questions for each employer based on the specifics of a given job. Some companies pay for parts of the tests to be customized just for their business.
As test results roll in, hiring managers get real-time access to a database that ranks job candidates based on their scores. The companies can then decide who to pursue for further interviewing based on that ranking.
“It’d be very common amongst our clients to screen about the bottom 30 percent,” Lahti said.
CEB’s Web-based tests have uses beyond screening prospective hires.
“There are companies now that actually distribute, often for an incentive like a gift card, a quick hit survey on a college campus,” said Jean Martin, executive director of CEB’s human resources practice.
Based on those survey results, the company then has a pre-screened group of candidates that it can go to campus and target.
At T-Mobile, head of talent acquisition Jared Flynn said the company is using assessments to determine who in its current workforce might a good candidate for a new, high-potential leadership program.
And in the future, Lahti sees the possibilities opening even wider. Technologies such as virtual reality, Lahti said, could be used to offer pre-employment testing that is even more immersive and lifelike.