Nate Silver is best known as a statistician and election analyst (psephologist) who correctly predicted the winner in 49 of the 50 states during the 2008 presidential race and called all 50 states correctly this past week. He quietly persevered in his election analyses despite a torrent of criticism and invective from a variety of commentators who called the race a tossup or even insisted that Mitt Romney would win handily.
Notwithstanding his track record, however, his book “The Signal and the Noise” is a much more general tome about predictions good, bad and ugly, whose basic outline is straightforward. In the first half, he examines predictions by experts in the fields of finance, baseball, politics, health, weather and the economy. In the second half, he discusses ways in which these predictions might be improved and how they might help clarify issues such as global warming, terrorism and market bubbles.
(Penguin) - ’The Signal and the Noise: Why So Many Predictions Fail-but Some Don't’ by Nate Silver
The strength of the book lies in the abundance of relevant detail Silver provides about each field and his analysis of why predictions are generally much better in some fields than others. He interviews a wide variety of knowledgeable people; an especially prominent source is psychology professor Philip Tetlock, who has amassed considerable evidence that most prognostication by professors, journalists and government officials is close to worthless, a conclusion with which Silver clearly agrees (as do I).
Even in the political domain, however, there are some bright spots. Silver and Tetlock mention essayist Isaiah Berlin’s reference to a poem by the ancient Greek poet Archilochus, who wrote that “the fox knows many things, but the hedgehog knows one big thing.” The predictions of pundits who are more fox-like, who stick to little facts, telling observations and small-bore issues, are usually somewhat more accurate than those of pundits whose approach is more like that of a hedgehog. The latter tend to try to fit everything they hear into the same tidy, overarching narrative, a tendency Silver notes is particularly prevalent on pundit-laden shows such as “The McLaughlin Group.” Silver’s book and his deservedly popular and impressively accurate FiveThirtyEight.com election blog for the New York Times reveal him to be a confirmed fox.
The ideological approach is related to other characteristics of poor predictions: ignorance of probability and, especially, overconfidence. In fact, in all the fields discussed, overconfidence is associated with underperformance and, I would add, with another common personality type: the extreme hedgehog, otherwise known as the hot dog.
Not surprisingly, data-rich fields lend themselves to better predictions. Baseball is one, as Silver well knows. His PECOTA system, which analyzed and predicted the career development of major league players, proved quite successful. In discussing statistician Bill James, general manager Billy Beane and others, he states that their successes were possible in large part because they analyzed reams of data, supplemented them with scouting reports, and were not loath to revise their criteria when they found some that worked better and were more predictive.
Loading...
Comments