The Psychic Octopus – Modeling Magical Oracles of Irrational Processes, A Comment on Pielke and Silver


Wygart has a comment in over at Pielke the Younger’s blog on his recent post Parlor Games and Predicting Presidential Elections in which he discusses an article by Nate Silver at the NY Times on the ability of political scientists to predict elections based upon measurements of various [mostly economic] “fundamentals”

Silver asks:

Can political scientists “predict winners and losers with amazing accuracy long before the campaigns start”?

And his answer

The answer to this question, at least since 1992, has been emphatically not. Some of their forecasts have been better than others, but their track record as a whole is very poor.

And the models that claim to be able to predict elections based solely on the fundamentals — that is, without looking to horse-race factors like polls or approval ratings — have done especially badly. Many of these models claim to explain as much as 90 percent of the variance in election outcomes without looking at a single poll. In practice, they have had almost literally no predictive power, whether looked at individually or averaged together.

Pielke the younger says:

“Ouch.”

And he goes on, in his usual inimitable way, to dissects the issue, establishes the parameters of what a ‘skillful’ model would have to accomplish in order to prove its, er, skillfulness, how modelers fool themselves and others into thinking that their pet model has some skill, and then lays out the actual track record – not good.

Continue reading