Auto reliability ratings might not be reliable with problems such as Toyota's

By Annys Shin
Washington Post Staff Writer
Saturday, February 6, 2010

Toyota's recall of millions of cars in order to fix sticky gas pedals and loose floor mats has inconvenienced customers around the world and damaged the automaker's reputation. Now it could also be undermining public confidence in the system of independent ratings and reviews that consumers have come to rely on and that for decades gave Toyota vehicles high marks for reliability and safety.

The consistently strong ratings Toyota vehicles have received over the years from Consumer Reports, and other consumer auto sites have fueled sales and helped the Japanese company surpass General Motors last year as the world's largest automaker. However, for many consumers, those ratings now appear to fly in the face of serious safety issues spotlighted by the sticky-pedal recall and a separate recall Toyota expanded last fall, aimed at stopping gas pedals from getting caught in floor mats. Both recalls stem from hundreds of complaints of sudden unintended accelerations in Toyotas that have allegedly been linked to more than a dozen fatalities since 1999.

Auto safety experts say consumers might need to adjust their expectations about ratings from private groups because of the limited nature of their testing and the degree to which they rely on government and industry testing that itself is in large part based on trust. And their recommendations are no substitute for proper surveillance by regulators and manufacturers.

Some of the most strident criticism has been reserved for Consumer Reports, which accepts no advertising and is among the most respected reviewers. On Jan. 29, three days after Toyota announced the latest recall, the magazine suspended its recommendations for eight models, citing concerns raised by the recall. Jim Guest, president of Consumers Union, the publisher of Consumer Reports, said in a statement it was doing so because "the vehicles have been identified as potentially unsafe" and "without a fix yet being available to consumers . . . our position is that you shouldn't compromise on safety."

The move sparked a backlash on the magazine's Cars Blog. One reader who identified himself only as Kevin wrote, "instead of giving an automatic 'recommend' rating to Toyotas, don't your think your readership (much of whom looks to you -- and only you) deserves thorough retesting of all Toyotas and revised ratings based on said testing?"

Another reader identified as John wrote, "Consumer Reports, you make me very irritated. Remove all recommended status of all Toyota vehicles not just temporarily, but permanently."

Safety Research & Strategies founder Sean Kane says consumer auto reviewers should not be faulted for not detecting problems that occur infrequently and often intermittently and whose causes have been hard to pin down. "They are evaluating vehicles in a different way," he said.

Kelley Blue Book offers reviews on its Web site but doesn't do any independent testing, said spokeswoman Robyn Eckard. does do some formal testing of new models, focused mainly on comfort and performance, said spokesman Chintan Talati. It also posts service bulletins, safety advisories and recall notices from automakers, information from the National Highway Transportation Safety Administration, and crash-test data from the Insurance Institute for Highway Safety. Commenters have raised the gas pedal issue in online forums but not in enough volume to raise red flags, Talati said.

Consumer Reports bases its ratings on three factors: a test drive on a closed track aimed at evaluating drivability and performance, reliability data gathered from a survey of car owners that covers 1.4 million vehicles, and crash-test data compiled by federal regulators and the IIHS.

David Zuby, IIHS senior vice president, says the institute's crash testing, regarded by safety advocates as more stringent than the NHTSA's, focuses on crash-worthiness and would also not be likely to catch problems with gas pedals or brakes. The institute provides separate ratings for front, rear and side-impact crashes, as well as for roof strength, particularly important in rollover accidents. It also awards a separate top rating to vehicles that rate well on the crash tests and also come with electronic stability control, which can help prevent accidents.

In 2009, Toyota won more IIHS Top Safety Pick awards than any other manufacturer; in 2010, it didn't win any. Zuby said he and his colleagues look at complaints filed with the NHTSA but not regularly, and when they do, they tend to focus on crash-related comments.

Consumer Reports attempts to suss out emerging problems using its vehicle owner survey. It looks for incident rates of at least 1 in 100, said David Champion, director of automobile testing for the magazine. The NHTSA has received approximately 2,262 complaints of sudden unintended acceleration in Toyotas since 1999, according to an analysis by Safety Research & Strategies, a vehicle- and product-safety research firm based in Rehoboth, Mass. Toyota sold about 20 million vehicles during that period, for an incident rate of 1 in 10,000.

Consumer Reports also looked at 2008 complaints of sudden unintended acceleration in the NHTSA database and found that Toyota had received more complaints relative to its market share than its competitors. Toyota vehicles triggered 41 percent of complaints of sudden unintended acceleration; the company's market share at the time was about 17 percent. However, demonstrating how rare sudden unintended accelerations are, the total number of complaints involving Toyotas in 2008 was 52 out of 2 million, Champion said.

Champion said the magazine's methods are limited by the fact that its main source of reliability data is a self-reported survey, in contrast to warranty claims and accident and injury reports that the NHTSA and manufacturers can access.

Randy Whitfield, who runs a Crownsville statistical analysis firm, Quality Control System, likens what Consumer Reports does to a small-scale clinical drug trial, which makes it unlikely it will uncover every potential problem that is liable to crop up when the drug is used by millions. "Everything changes when you put something out in the field," he said. Whitfield's firm, which has done work for Consumers Union, said a better surveillance system is needed to track problems such as sudden unintended acceleration.

Sudden unintended acceleration is most likely a "multi-root problem," Kane said, that is more complicated than the simple mechanical explanations offered by the manufacturer.

In 2007, Toyota said the problem was gas pedals getting caught in floor mats and recalled a limited number of models. Then last November, it announced it was recalling more than 4 million vehicles due to gas-pedal-snagging floor mats. That was followed by the latest recall, involving sticky gas pedals. At least four class-action lawsuits also allege the vehicles' electronic throttle-control system is to blame.

© 2010 The Washington Post Company