The Irrelevance of Polling in the Democratic Primary

Darn it if Hillary Clinton isn't going to go out like that dude careering off the side of the high-rise at the end of Die Hard. Today brought another parting shot: a 68-32 thumpin' in Puerto Rico.

Once again, the polls proved relatively useless at forecasting this one-sided result. The two pre-primary polls showed Hillary ahead by an average of 16 points, less than half the eventual victory margin.

One of the real stories of this primary has been how limited polling has been as a tool, and how Democratic electorates in state after state have defaulted to demographics. At critical moments in the primary season, momentum has shifted based on movement relative to the final polls, when that movement was simply an expression of the underlying demographic trends in that state.

  • Final polling in New Hampshire was so cosmically wrong that a special committee of the American Association of Public Opinion Research was formed to look into the discrepancy. We now know that Obama has struggled in blue-collar, suburban New England -- losing MA and RI handily, and winning the Maine caucuses by about half his usual caucus victory margin. The demographics slightly favored Clinton, the polling favored Obama, and demographics won. 
  • South Carolina is the linchpin of the argument I'm about to make. That was the state that disrupted the narrative, transforming a likely loss into a Super Tuesday stalemate that created the situation we are in today. The RCP average showed Obama up by 11.3. He won by 28.9, thanks to the black vote. This sent shockwaves through the political system. But it was perfectly consistent with nearby states, with Obama winning Georgia by 35 percent and North Carolina by a greater than expected 14.7 percent. It should be noted that the gap between results and final polling was much higher in South Carolina (as well as Georgia and Alabama). Polling's low-balling the nearly unanimous black vote -- normally not a factor with the usual all-white field, created the conditions for Obama to beat expectations and to finally tie Hillary in national polling ahead of Super Tuesday.
  • The first post-Super Tuesday stop was the Potomac primary. Here, Obama took Virginia by 28.8 percent when the final RCP average predicted 17.7. The outsized influence of Northern Virginia, the black vote in the south side of the state, and a small Appalachian rural influence conspired to make it so. This was the first confirmation of Obama's post-Super Tuesday momentum, leading to his 10-in-a-row wins.
  • Virginia's influenced paled in comparison to Wisconsin. This was expected to be tight, with Obama narrowly ahead in the RCP average by 4.3. Instead, he blew the doors off with a 17.4 point win, fueled by proximity to Illinois, the state's progressive heritage, and lax voter registration requirements. This primary solidified his role as the favorite and was a key morale booster as the final big primary before "Super Tuesday 2" on March 4th.
  • North Carolina crowned Obama the presumptive nominee. Here, Obama outperformed the final polls by nearly 7 points. By now, the map was filled out enough to know that with 25-point-plus wins directly to the north and south, the predicted 8 point margin was just too low.
  • The final states that solidified the Appalachian + Hispanic narrative -- West Virginia, Kentucky, and Puerto Rico, saw greater than expected Clinton margins -- by 7, 7, and 20 points respectively. These primaries were immune to national polling showing Obama pulling away as the likely nominee. It's difficult to see how Clinton would have done better had these contests been held earlier in the process.

I've posted a spreadsheet with the calculations underlying this analysis. In a total of five primaries -- Puerto Rico, South Carolina, Georgia, Alabama, and Wisconsin -- the absolute gap between the final polls and the results was greater than New Hampshire, which is seen as one of polling's greatest historical failures. Of these, the momentum shifts in South Carolina and Wisconsin were crucial to determing the nominee.

In 25 of the 31 primaries analyzed, the trend between the polls and the final results benefited the winner. This is a bit tautological -- until you consider there was a pro-winner shift of 5.76 points across all contests. This is inside the margin of error of most polls -- but barely.

Overall, the polls showed an average error of 6.89 points compared to the results.

Results like this would seem to indicate primary results that were unpredictable and all over the map. Except they weren't. Now that we are approaching a perfect 50-state electoral breakdown of Clinton-Obama -- marred only by the presence of small-state caucuses -- we see that the results are totally internally coherent based on demographics. In fact, I'm prepared to argue that we would have gotten almost the same margin in each state in a national primary held February 5th. Because the candidates themselves polarized the electorate demographically, momentum and events (and polls) made little difference.

The polls' vulnerability was exposed in part because of the unique nature of the Democratic contest. Usually, we only pay attention to polls in close races in diverse states with lots of countervailing trends. And in fact, the polls did relatively well in large states like Ohio, Pennsylvania, Texas, Illinois, and New York -- the glaring exception being California. But in a sequential process where even the most far-gone states matter because of the delegate split -- getting it wrong in the Deep South or Appalachia really impacted the narrative of the race.

For whatever reason, the polls seemed to do worse in states with large monolithic communities favoring either Clinton or Obama. Perhaps this is the "Wilder effect" -- professing to be open to the white woman or the black man to an interviewer but in the end voting for "our guy" -- or gal. 

Either way, I think this means that polling in the future will have to be supplemented with a heavy dose of Poblano-style Moneyball analysis.

4.333335
Your rating: None Average: 4.3 (3 votes)

Comments

68 - 32 in Puerto Rico. . .

. . .is a pretty serious drubbing.  When did Puerto Rico have an influx of bitter, gun-totin', xenophobic bigots?

Obama has fared very poorly

Obama has fared very poorly with Spanish-speaking households all season. It accounted for his sound defeat in California.

Hmm. . .

. . .isn't it odd that the media haven't made a point of highlighting that particular weakness, as opposed to working class Appalachian whites?

Of course, I was aware of Obama's difficulties with Hispanics.  I just find it a little amusing that the usual suspects choose to focus on the one demographic that Republicans are not under any circumstances allowed to acknowledge.  Dog whistle racism, and all.

They have focused on it where

They have focused on it where it was relevant, but there haven't been any large Hispanic populations at the polls since Texas. For both Texas and California the media played the story like a broken record, and indeed, the Hispanics almost singlehandedly carried California for Clinton.

I typed up a very long

I typed up a very long response to this post, but my browser ate it.

I've spent far too much time following the Democratic numbers this year. I've been oddly fascinated by the whole Democratic primary mess, with any obsession I usually reserve only for the BCS. (And the two systems are remarkably similar in their unpredictabililty, inefficiency, and ineffectiveness.)

I'm not going to rehash my original reply. But suffice it to say, I'm not sure irrelevant is the best characterization of the polling this season. Inaccurate, yes; but irrelevant, no. In spite of the inaccuracies, we've been able to gauge each state's trend headng into its primary pretty well. As you mentioned, there were a couple of glaring exceptions. But overall, the polls have remained relevant in that sense.

Looking forward to the general election

would this suggest we pay more attention to local demographics than the bouncing opinion polls?

For example, we are running more strongly than expected in MI and weaker in CO. Can this to attributed to hard numbers related to education, age, ethnicity, source/amount of income et al.?  

my thoughts...

The problem is not as simple as demographics, because there's the complication of figuring which demographic shows up at the polls. In most states this year, we've seen nearly the entire black registered population turn out to vote Barack. Yet in California it was the Hispanics who turned out en masse, and not the state's considerable African-American population. So while we obviously will be looking at each state's demographic profile, that still won't be an infallible predictor. (Not that you're suggesting it was.)

The Democrats are hurting in Michigan due to their primary mess there. Barack is particularly unpopular because he didn't support the revote proposal in the Michigan legislature. Colorado's a bit of a mystery. Polling there has been infrequent, and as a caucus state we don't get much info from the primary. Colorado has a strong evangelical element, with Dobson and other leaders based out of Colorado Springs. Colorado also has a growing Hispanic population, which will likely break Republican unless Obama goes with Richardson.

I really believe the

I really believe the Democrats' reliance on identity politics has come back to bite them in 2008.  Had the nomination come down to Hillary or Barack versus a white male, things might not have gone on this long, as so much of Democratic politics comes down to which aggrieved demographic with which a given voter identifies himself or herself.  It was obvious from the moment she announced her candidacy that Hillary envisioned her candidacy as a woman's turn (remember "shattering the ultimate glass ceiling?"), and in my opinion this demonstrated a keen understanding of internal Democratic politics.  Unfortunately, she faced an opponent who represented another identity group, and it ended up splitting the party in ways we may not appreciate until after the election.

Now the Democratic party, after years of politics basing appeals on race, gender, interest group, etc. (rather than unifying ideology) are in a pickle.  If Hillary loses, a noticable chunk of her supporters will cry sexism (as many already have), and were Obama to lose, you can guess where that might go as well. 

I don't know where this ultimately leaves the Democrats, but I do believe identity-based anger (most likely on the part of women after Hillary bows out) will damage turnout for their nominee in November, and if properly exploited, may even drive some new voters to the GOP or a third party.

I don't have a client right now , but if I did, I'd be looking very hard at what I might do to attract angry women to the party, much as Nixon did with angry southerners in 1968. 

I don't have a client right

I don't have a client right now , but if I did, I'd be looking very hard at what I might do to attract angry women to the party, much as Nixon did with angry southerners in 1968.

You are wise. I've been thinking about what McCain might do to pick up this vote. Do you think choosing a female VP would help? I can't decide whether it would help pull these women or only further affront them by reinforcing the notion that women are only deserving of the #2 post.

I think if messaged correctly

I think if messaged correctly putting a woman on the ticket might be a very good idea.  I think it would be important to reinforce that in choosing said candidate, the criterion McCain primarily used was that that person was qualified and ready to be president, should something happen to him, or when he leaves office.  In other words, not a second fiddle, but a presidential understudy.  Of course, in doing so you run the risk or reinforcing McCain's age. 

There's been a lot of talk about McCain putting a minority on the ticket.  While admirable in itself, I don't think it's a good strategic move.  Doing so would essentially be an attempt to "out-minority" Democrats, in other words trying to outdo them on one of their strengths, rather than exploiting one of their weaknesses.  I think picking a woman accomplishes the latter.

Of course, it would be nice if we could just pick our candidates based on their merits, judging them not by the color of their skin (or their gender) but by the content of their character.  Wouldn' t that be great?

Of course, it would be nice

Of course, it would be nice if we could just pick our candidates based on their merits, judging them not by the color of their skin (or their gender) but by the content of their character.

 Pshaw. That's just crazy talk. Choosing candidates by experience and character. As if.

I agree with you 100%. If the GOP puts a minority on the ticket, it looks roughly like a little kid yelling "Me too! me too! We've got black people too!"

A woman VP I think could be more effective on the GOP ticket than one on the Dem ticket, because on the Dem ticket it really does come across as a sort of consolation prize, whereas on the GOP ticket - if handled correctly - it could really be seen as a just accomplishment based on the woman's hard work and experience. Still difficult waters to navigate, though: If the woman is too young she'll be seen as leapfrogging the Baby Boomer women who worked so hard to pave the way for my generation. If she's too old she may not offset McCain's, well, oldness. And then there's of course all the policy issues that you'd logically think would also be a consideration in selection.

 

Just had another thought:

Just had another thought: McCain ought to share credit for some piece of his platform with the female running mate, the way a president might farm some policies or initiatives to the VP, but make them something that's a known part of the campaign.  I think this helps send the message that we're not just painting pretty pictures with women, but that we actually want women in leadership positions.  It also has the added benefit of saving McCain and his team a little bit of work, i.e. "I've always said that I'm pro-green fuzzy.  Our campaign has just released an exciting new green fuzzy initiative, under the leadership of my running mate." 

Oooh...what if she even had her own website -- coordinated with his, with the same logos and linking back to a lot of the same places, but making her a campaign personality in herself.

The concern I have here, though, is where the bench is for these candidates.  Condi Rice, while certainly impressive in her own right, has never held elective office and is inextricably tied to George W. Bush.  Kay Bailey Hutchison is too old.  Carly Fiorina is intriguing, but once again I'm concerned by the fact that she's never held elected office -- the presidency and vice presidency are not for amateurs.  But I think in terms of temperament and image, the pro-Fiorina crowd may be on the right track.

Righto. Therein lies the

Righto. Therein lies the complication. Girl runningmate is good in theory, but which girl is good in practice. (Note to McCain: avoid referring to running mate as "girl.") Condi is associated with Bush, and, further, wants to retire from Washington. Hutchison is older and from Texas - we don't need no stinkin' Texans. Fiorina is a step in the right direction. She was smart enough to jump ship from Lucent before they tanked, so she's clearly got some business savvy that certainly wouldn't hurt McCain's ticket. This would play well into your suggestion that he credit her with certain policy proposals...he's weak on the economy and she could step up. I must say I don't know much about her policy leanings, though, and she does lack name recognition. Palin would  play well with social conservatives but she's so young she might not appeal to the older women who formerly supported Clinton. I can't think of anyone else right off hand, though I'm sure there are others.

Maybe someone along the lines. . .

. . .of Abigail Thernstrom.  I don't know that she specifically would bring everything needed to the ticket, but as a template, given her academic background and the issues that will likely arise in the course of the campaign, she would be well suited for the job.  Unfortunately, since she is an academic, you have to wonder how well her obvious plusses would translate into the political world. 

I don't know her age, and she doesn't address the economic issue.  But, she's an undeniably brilliant woman that I think conservatives across the board could get behind.  It's the independents that I'm not so sure about.

It's just an off-the-wall suggestion, as long as were kicking around the idea of a woman on the ticket.

Interesting. She's really old

Interesting. She's really old (older than my parents counts as really old), but you have a point that a highly intelligent woman rooted in academic thought and activism and strong in debate would do well. McCain...well...debate and academics don't seem to be his strong suits.

Our target voter is a nurse in Livonia, MI

and you guys think an academic running mate is the answer?

I think the answer is a

I think the answer is a 45-year-old woman who has worked her way up a ladder somewhere, having to prove herself to men every step of the way while simultaneously raising a family. Whether her experience is in business or academics won't matter to the nurse in Michigan, so long as she identifies with the struggle.

Actually, no. . .

. . .I think it could potentially be problematic in some ways.  But, as Lisa pointed out, a woman who has bootstrapped her way into prominence would be great.  Unfortunately, academics are easily protrayed as out-of-touch and isolated from the real world.

Still, I find it hard to believe that there isn't a woman out there who is conservative, young, and can boast the kind of brilliance that Thernstrom does and has translated academic achievement into real world results in some way.

Incidentally, I don't think the nurse in Livonia would necessarily be a tough sell for a woman of academic achievement like Thernstrom.  The stay-at-home mom and the legal assistant who has trouble putting gas in her tank would be more problematic, though.

Pro Choice Women Republican

I'm sure conservative's would be pissed off if McCain picks a moderate Republican, but i think the only way he can win is to run to the center. Are pro life people really going to vote for Obama, who supports partile-birth abortion, refuses to condemn infanticide of failed late term abortions, and would appoint a bunch of Ruth Bader Ginsburgs to the supreme court.

There are some quality female moderate Republicans that are pro choice. Olympia Snowe comes to mind. Very qualified, popular and experianced. About 60 years old. I think that would boost McCain chances significantly.

In 2007 <a

In 2007 <a href="http://www.linksjewel.co.uk">links of london</a> was named  Jewellery Brand of the Year at the 2007 UK Jewellery Awards for the third consecutive year.links of london specialises in quality handcrafted designer stering silver jewelry, links of london Shop provides hundreds of discount, cheap and links of london  Tiffany<a href="http://www.linksstore.co.uk/linksoflondon-earrings-19.html">links of london Bracelet Earrings</a><a href="http://www.linksstore.co.uk/linksoflondon-friendship-16.html">links of london Bracelet Friendship</a><a href="http://www.linksstore.co.uk/linksoflondon-necklaces-14.html">links of london Bracelet Necklaces</a>Links of London, the leading British contemporary jeweller was founded in 1990 by jewellery designer Annoushka Ducas and her husband John Ayton. Excellent craftsmanship, generous tactile forms and the use of the finest materials remain integral to each collection.<a href="http://www.linksstore.co.uk/">links of london store</a><a href="http://www.linksstore.co.uk/">linksoflondon</a><a href="http://www.linksstore.co.uk/linksoflondon-bracelets-13.html">links of london Bracelet</a><a href="http://www.linksstore.co.uk/linksoflondon-charms-15.html">links of london  Charms</a><a href="http://www.jewelrytiffany.co.uk">tiffany jewellery</a>

 

The major problem with

The major problem with Sowell’s post is to claim the risk would be greater under Obama. The risk of terrorists getting nuclear weapons cheap tiffany jewelry