The Washington PostDemocracy Dies in Darkness

Athletes can easily trick popular concussion test, study finds

Ohio State offensive and Rutgers defensive linemen go helmet-to-helmet on a play during a 2017 college football game in Piscataway, N.J. (Mel Evans/Associated Press)

Duping one of the nation’s most common concussion screening tests is significantly easier than previously measured, according to a new study conducted by researchers at Butler University.

The researchers found that half of test takers who “sandbagged,” or purposefully underperformed, on the Immediate Post-Concussion Assessment and Cognitive Test, known as ImPACT, went undetected. That means those test takers were able to fool the exam, and thus might have returned to physical activity sooner than medically appropriate.

Athletes generally take ImPACT before the start of their sports season to record a baseline score. The test, first released in 2002, has been administered more than 12 million times, according to ImPACT Applications, the company that makes the exam.

The company has deals to provide the assessment to Major League Baseball, Major League Soccer, U.S. Lacrosse and the British Football Association. The test is also commonly administered to high school and youth athletes. ImPACT markets a pediatric version of the exam, as well.

The computerized test asks subjects to answer word, design, color and symbol matching questions to test the brain’s function. After a traumatic brain injury, an athlete is asked similar questions in another version of the test, and the scores are compared.

If an athlete “sandbags” on the original baseline assessment, he or she would only have to match that lower threshold to possibly return to action after a head injury. Nearly a third of athletes reported to researchers in a separate 2017 study that they didn’t provide “maximal effort” on computerized neurocognitive tests, such as ImPACT.

“Anyone who works with the concussed clinically knows there are a lot of people who purposefully sandbag the baseline test and a lot of people don’t get caught,” said Amy Peak, one of the study’s authors and director of undergraduate health science programs at Butler. I would hear in the community and hear all these athletes tell me, ‘I sandbagged mine, everybody sandbags it.’ ” 

Previous studies, including one cited by ImPACT’s “Administration and Interpretation Manual,” have found that the test successfully flags 89 percent of sandbaggers and invalidates their scores, which requires subjects to retake the baseline assessment. The test has five built-in indicators meant to flag any irregularities.

But Butler’s study, co-researched by Indiana University medical student Courtney Raab, found only two of those indicators detected more than 15 percent of test takers who tried to trick the exam. Peak found half of test subjects were able to defeat the flagging mechanisms without being coached.

Peak and Raab presented the findings at the American Academy of Neurology Sports Concussions Conference in Indianapolis this month.

“If you believe the built-in invalidity indicators in this test are going to flag the people who are flunking on purpose, they’re not,” Peak said.

ImPACT Applications did not immediately respond to requests for comment.

Peak recruited 77 volunteers for the study, 40 of whom were randomly chosen and told to sandbag the test, and 37 of whom were told to try their best.

None of the 37 volunteers in the control group were flagged for invalid results, as expected. But 20 of the 40 sandbaggers successfully fooled the test.

Peak said test proctors, often a school or team’s designated athletic trainer, can reduce sandbagging by reinforcing to test takers the importance of recording an accurate score and the grave consequences of repeated brain injuries.

In many cases, proctors know test takers personally, Peak said. They shouldn’t be afraid to confront athletes if their ImPACT score does not match with their academic or on-field performance.

“Once you know the individual, that should give you a strong gut feeling,” Peak said, “Especially in colleges, these trainers know athletes well. They should be able to tell if something doesn’t look right.”

Read more from Post Sports:

UNC’s Larry Fedora, who questioned football’s link with CTE, finds backers at Fox Sports and Yahoo

UNC Coach Larry Fedora criticized after questioning football’s link with CTE

Brewer: Larry Fedora’s defensive crouch is as dangerous to football’s future as concussions

More claims mean NFL concussion payout could jump by $400M

NCAA concussion case settles three days into trial