In Sea of Data, Not All Numbers Are Equal

We need standards, not averages. There's certainly a place for averages. My investment portfolio, for example, would be in better shape today if I had invested in broad indexes of securities instead of fancying myself a stock-picker. At the same time, I'd be in a much tighter financial position if I took investment advice from spam e-mails as seriously as that from accredited financial experts.

This last point exaggerates the disparities among pollsters. But there are differences among pollsters, and they matter.

Pollsters sometimes disagree about how to conduct surveys, but the high-quality polling we should pay attention to is based on an established method undergirded by statistical theory.

The gold standard in news polling remains interviewers making telephone calls to people randomly selected from a sample of a definable, reachable population. To be sure, the luster on the method is not as shiny as it once was, but I'd always choose tarnished precious metals over fool's gold.

Before anyone feels condemned to night classes and bell curves to sort through the glut of polls, let me say that the primary filtering burden should rest with the news media. It's ironic in a field that prides itself on sorting reliable sources from bogus ones that so many treat all numbers -- including poll estimates -- equally, and as valid on their face.

News organizations should be aware that they give immediate credibility to the "facts" they air or print. But this is not an argument for some sort of media monopoly on polling information.

To the contrary, an ancillary burden is on survey research organizations to openly disclose how and why they ask the questions they do, so people can judge for themselves whether to believe the results.

Pollsters should disclose the sponsorship of their polls, provide a detailed statement on how they conduct their research and reveal their full questionnaires. Few will be interested in reading all this, but it should be readily available. So many polls are part of political or public relations campaigns that it's particularly important to be able to check for signs of bias.

News organizations should carefully review all such information before putting their names on polling data by publishing it. Most people have finely tuned nonsense filters, many have changed their computer settings to avoid countless e-mail offers, and it's high time for the news media to set up poll filters.

Grandma has a nice Southern expression for the current alternative, something about a pig and lipstick. And as one of the aggrieved pollsters she derided last week, I'll try to leave you alone for a while so we can recover from any lingering poll fatigue.

Cohen is The Post's director of polling.