In my previous post on Kos and Research 2000 I noted how weird it was that Kos could afford to commission dozens of campaign polls given that by (his own admission) he runs a low seven-figure operation and the polls are likely far from his main traffic driver -- though I'm sure they don't hurt eyeballs-wise.
I put this question to a pollster, who said R2K's claimed methodology and the likely cost of doing such polls legitimately raised immediate red flags. The pollster pointed me to this massive, 2000-person survey limited to Republicans back from January, commenting thusly:
Take, for instance, their large January 2010 survey aimed at proving Republicans were all kooks. They did a sample of about 2000 Republicans - a totally absurd sample size, most pollsters wouldn't in good conscience have a client pursue a survey of that size unless they had microtargeting aims and really needed a lot of subsample detail. Unless you really, really want a big sample for the smaller cells (say, you want 100 interviews from female Hispanic Republicans age 18-34) there's no reason to do a survey of that size. 1000 interviews will do for a national. Sometimes we go up to 1250 with our bigger clients who really need that level of detail on a few key subsamples. The difference in margin of error from 1000 (+/- 3.1%) and 2000 (about +/- 2%) is not a huge deal, not worth spending 2x as much on a poll.
Now, even weirder, it is just of Republicans, AND it wasn't done from a listed sample. When you want to do a survey of, say, primary voters in a statewide, a listed/voter file sample is a totally acceptable practice because the alternative is unbelievably costly. Think about it - not only did they call 2000 people, but they randomly dialed people, and turned away anyone who didn't identify as a Republican. This will crush your incidence rate (meaning the number of folks who pick up the phone who are eligible to take the survey) and send costs through the roof. We're talking at least tripling the costs.
A survey of the length of that January 2010 survey, about 25 short-ish questions, plus a handful of demographics, is probably about a 10 minute questionnaire (I'm just eyeballing it and assuming an introductory statement and guessing on the # of demos asked, I could be off by a few minutes). Fielding a 10 minute questionnaire to 1000 registered voters is going to run you in the $25-30 range. Fielding a 10 minute questionnaire to 2000 voters? Probably 45-55. But with the crazy drop in incidence caused by the Republican screener? That survey could not have been done for less than six figures. Period.
Remember now that Research 2000 never claimed to be a robo-polling outfit. They claimed they did live interviews. And most polls are of likely voters, not registered voters. More screening means more cost. As far as what R2K claimed was its methodology, we're pretty much talking the Cadillac in terms of what the polls should cost.
So, the question is did Kos really pay high five-figures, or low-six figures, for a single poll to drive eyeballs to one or two blog posts to prove Republicans are nuts? Huh?
I'm guessing no. I'm guessing R2K sold it to him for far less, say $10,000? And anyone with a rudimentary understanding of polling would have known you can't do a poll like this for that amount of money. So the question now is what this says about what Kos should have known about this. Is he so rich he can drop 100K on a single poll to drive a single day's news cycle -- something not even the major networks would do? Is he simply gullible? Or was he negligent in not checking out what what I can only guess were R2K's absurd price quotes compared to live operator pollsters?
It wasn't just (relatively) deep-pocketed new media sources like Daily Kos who were spending money on Research 2000 polls. R2K did polling for state-level liberal blogs like Blue Mass Group in the run up to the Massachusetts special election. On January 14, R2K produced a poll showing Coakley with an 8-point lead (while other polls were showing Brown pulling ahead), and in touting the "good" news, Blue Mass Group proudly noted that "Research 2000 does live interviews, unlike robo-pollsters Rasmussen and PPP." My polling source had this response:
A simple ballot test and a handful of demographics wouldn't be very long. But even if that was only a 4 minute survey, you're still talking at least at least 6-8 grand for the raw interviewing costs without any additional markup.
Did a Massachusetts progressive blog pay more than $6,000 for a top-of-the-line survey when maybe a half dozen other pollsters were polling the race by that point? Really? This begs the question of what Blue Mass Group really paid. And what did Kos really pay? And if the numbers are within what seems like their modest budgets (by mainstream media standards), it should have raised red flags if they did any shopping around for other pollsters.