Jay Cost has recently had two good posts up about the recent polls, here and here. They’re both well worth the read, but the gist is that the polls are showing variance that can’t be explained just by sampling error.
Instead, pollsters seem to have two different views of what the electorate will look like. Indeed, most of the movement that has occurred in the polling average can be explained by different pollsters entering and exiting the average. Last week the average was at roughly a six-point Obama advantage, but that quickly changed with the addition of the NBC/WSJ poll and the CBS/ poll. The polls themselves didn’t change much (the seven tracking polls that were in the average at this point have barely moved in their avaerage), but the pollsters in the average have.
So what we’ve seen is some pollsters, such as IBD/TIPP and Battleground have fairly consistently showed a 3-point race or so. Other pollsters, such as CBS/NYTimes have consistently been at the high end of the spectrum. Rasmussen has more consistently been in the middle, with an Obama lead of 6-8 points.
We see a similar effect in some states. Florida, Minnesota, New Hampshire, and Ohio all have had a number of polls in recent weeks outside of each others’ error margins. Rasmussen and Quinnipiac have consistently been on the opposite ends of the Ohio spectrum. This is complicated by the fact that individual states tend to be less frequently polled than the nation as a whole, and tend to have less reputable pollsters or less experienced pollsters in them (Big 10 Battleground anyone?), and that their polls tend to have smaller samples, making it more difficult to sniff out outliers.
The upshot of this is that I’m not sure polling averages will be all that useful this year. Poll aggregation assumes that the pollsters are working off of basically the same script, and that you can thereby treat them as a single giant sample. This has the advantage of cancelling (or at least reducing the effect of) the outliers.
But when the pollsters have fundamentally different views of the electorate for their models, you can’t do this anymore. If one pollster thinks there will be a massive upsurge in the youth vote, and one pollster doesn’t, and they weight their polls accordingly, then they aren’t working off the same script. This is always somewhat of the case, but I think it’s more pronounced this time, given substantial uncertainty as to the eventual makeup of the election. As I said, given the number of apparently outlying polls that Cost identifies, I think that’s the only conclusion we can draw. Pollsters just aren’t polling the same election. You may as well try to aggregate the North Carolina and Virginia polls in order to predict Virginia.
Back in August, I said that the million dollar question this election was the makeup of the electorate I think that’s still the million dollar question. If IBD/TIPP or Battleground have the correct model for the electorate, then we can guess that the state polls showing the closer race are the correct ones. Given that IBD/TIPP and Battleground don’t poll states, we might even see several results to the right of what all the most pro-McCain polls are showing. And if that’s the case, a comeback and/or a win for McCain is still possible.
If, on the other hand, Pew has it right, McCain is probably down ten in Ohio, down seven in Florida, down eleven in Virginia, and down five in North Carolina. In which case, it really is all over but the shouting. And we can be certain there will be plenty of that.
But the bottom line is that this isn’t likely to be an election where all the pollsters can write off their error as being the error margin. Someone is going to be really, really wrong.