This story has been updated.
Your fitness tracker may be accurately counting your steps but not the correct number of calories burned, according to a new report by the Human Performance Laboratory at Ball State University in Muncie, Ind.
The study, which will be published in Medicine & Science in Sports & Exercise later this summer, found that FitBit and Jawbone are significantly over- and underestimating calories burned during certain physical activities .
The group’s findings come at a particularly inopportune time for Fitbit, as the company is currently facing a class-action lawsuit alleging its product, specifically its heart rate technology, is faulty and inaccurate.
“When you look at the literature, there is really not a lot of information on how well these wearable technologies work [or] how well they track variables that they are displaying on their apps,” says Alex Montoye, a co-author of the study and an assistant professor of clinical exercise physiology at Ball State University. “I was a little bit skeptical, just because I know how hard it is to measure activity even with the research-grade monitors.”
His skepticism inspired him to evaluate how these new consumer-based activity trackers stacked up to his team’s medical equipment. For the report, his team had 30 healthy adults, of varying ages and fitness levels, perform 10 different activities of varying difficulty for a short amount of time. They monitored the participants’ progress with a portable metabolic analyzer (to get a true, breath-by-breath analysis) and four different activity devices: two wrist-worn trackers (Fitbit Flex and Jawbone UP24) and two hip-worn trackers (Fitbit Zip and Fitbit One).
FitBit and Jawbone were chosen because in 2013, when the group was in the advance planning stage of the study, the companies “occupied about 85 percent of the wearable technology market,” Montoye explains.
The differences in calories recorded by the wearable technologies vs. how many were actually burned according to the team’s equipment were considerable. On average, the amount of energy exerted, or calories lost, was overestimated by 16 to 40 percent during ambulatory activities, such as walking, jogging and climbing stairs, and underestimated during household activities, including vacuuming, gardening and sweeping, by as much as 34 percent.
According to the report, the calories burned while walking were overestimated by three out of the four monitors by 26 to 61 percent. Cycling was the worst activity recorded; all the monitors significantly underestimated the number of calories burned by 37 to 59 percent. “In these activity monitors, the main technology in them is an accelerometer, which tracks changes in movement,” Montoye says. “So, if you’re sitting on a bike and your arms are stationary and your hip is stationary, they aren’t going to record any activity.”
Montoye stressed that the difference could be partly due to the limited amount of time recorded for each activity, as subjects were only monitored for five to 10 minutes. He also discussed how the location of the fitness tracker (hip vs. wrist) and how the device is worn (loose vs. tight) can affect its accuracy.
The most accurate estimates of calories burned were during sedentary activities, such as typing on a computer, watching television or writing. “I think that [is] the most surprising finding, to me,” Montoye says. “I expected, you know, if the wrists are moving, at least for the wrist monitors, that the devices would have picked up steps. But they did not.”
Fitbit released the following statement addressing the report. “Fitbit trackers are designed to provide meaningful data to our users to help them reach their health and fitness goals, and are not intended to be scientific or medical devices,” they wrote in an email to The Washington Post. “Our everyday trackers, such as the Fitbit Flex tested in this study, do a great job calculating calorie burn for step-based activities. When using a Fitbit tracker with PurePulse continuous wrist-based heart rate tracking, available on newer products . . . a user is provided with better calorie burn for non-step based workouts.”
The report did find that activities with a fairly well-defined motion, such as stair climbing or walking or jogging, counted the number of steps taken extremely well, within about 1 to 2 percent.
But if steps are the only mechanism that is being accurately reflected by these devices, should people just invest in the average, and cheaper, pedometer?
“The pedometer certainly is cheaper, but you sacrifice a lot with [it] as far as if you like having a smartphone app or a community where you can see what your friends are doing, setting goals and tracking each other’s progress,” Montoye says.
His advice? Don’t ditch the device just yet, but do take the tracker’s readings with a grain of salt. Think of them “more as a ballpark number,” Montoye says.
From from Wellness: