Daily Kos has sued Research 2000, its former pollster, for fraud. On the surface, the allegations seem a lot like the case Nate Silver made against Strategic Vision. In essence, when you're making up the numbers, odd biases and consistencies tend to creep in. You tend to favor certain numbers over others. The crosstabs, even on ridiculously small sub-samples, look too "clean." The report detailing the allegations is here.
The one R2K poll on a race that I was working that now seems to make perfect "sense" in light of this new information is the poll of the California Senate race two and a half weeks before the primary that showed Tom Campbell building a 15 point lead in the GOP primary while polling on adjacent field dates showed Carly Fiorina building a 20 point lead. I recall thinking that if there had genuinely been 35 points of movement in 48 hours (absent some major cataclysmic event, which there hadn't been), that'd be virtually unprecedented in the history of polling. If one were to make up a poll lead, a 15 point Campbell lead made sense if one looked at the past movement in the polls, but not in terms of what was actually happening on the ground at that point. It all makes a lot more sense now.
A lot of folks are trying to point to the root causes of this seeming debacle (including slamming robo-polls, which I think is off-base given the accuracy of outfits like SurveyUSA and PPP) but it will be interesting to see what the coming lawsuit(s) reveal about the relationship between Daily Kos and R2K. R2K was around prior to Daily Kos, and my vague recollection is that there was nothing out of line about its polling prior to its Kos contract. I could be wrong, but their polls seemed to play it up the middle. When an R2K poll came out in a previous election year, I didn't automatically assume a Democratic skew like I would a CBS/New York Times poll or a Newsweek poll. Yet the moment they signed up with Kos, all their results seemed to skew towards Obama and Congressional Democrats, starting with their 2008 Presidential tracking poll. Their 2010 polling was if anything worse, skewing several points toward Democratic Senate candidates, though their numbers in primaries seemed right, at least until the end when they disintegrated upon close contact with actual results.
At some point when I raised this previously, it was mentioned that R2K was simply assuming a turnout model closer to Obama 2008. If so, who would be pushing them to do that? R2K? Or Kos?
Did Markos tell R2K to produce fraudulent polls showing Democrats up? Clearly not. Could R2K have simply been too eager to please their client, producing skewed results and making stuff up to boot? That seems more likely. Either way, R2K's newfound pro-Democratic skew had the effect of skewing the polling averages in a way that even Strategic Vision (which performed "better" on 538's Pollster Ratings) didn't.
Another question to me is the volume of polling that R2K produced for Kos. I have a hard time finding a pollster who was this prolific for an individual client as R2K was for Kos. SurveyUSA has been equally if not more prolific in past cycles -- though not so much this one -- but their clients are different all over the country, usually local TV affiliates. Likewise, pollsters like PPP (D), another highly regarded automated polling operation, will release polling as a promotional vehicle for themselves -- and will potentially pay for it by picking up political clients, or selling questions on a survey otherwise deemed for public release.
The bottom line is that polling, even automated polling, is expensive, and especially at the volumes R2K and Kos were doing. It's hard for me to believe that Kos's polling bills wouldn't have run into the deep six figures, which seems like an awfully big chunk of his $1 million (give or take) in revenue. I'm not the expert here, but it seems to me that more deep-pocketed media organizations haven't commissioned nearly this much polling (national networks release like, what, once a month?). Perhaps the unit cost was getting to be too low, and R2K's margins were getting squeezed by their arrangement with Kos, so they simply made it up. Either way, the damage to the credibility of the polling industry and the polling's effect on conventional wisdom, was fait accompli.
UPDATE: Research 2000 claimed they did live interviews, and were not robo-polling. Live interviews are naturally more expensive. Which means they must have pitched Kos on a ridiculously low cost per poll. Was this not in itself a red flag?