One More Thing On That PA Keystone Poll

Over at DailyKos, Adam B argues that the F&M poll that I blogged about yesterday is actually biased to favor McCain, rather than Obama.  His argument is this:

The Keystone Poll has the following regional sample:

I now have a final few questions for statistical purposes only.
CNTY. Region of state (What is the name of the county you live in?)

35% Central
15% Southeast
14% Northeast
11% Allegheny
10% Southwest
8% Northwest
7% Philadelphia

Here, by way of comparison, is the 2004 CNN Exit Poll breakdown of the Pennsylvania vote:

VOTE BY REGION  

Philadelphia (12%)
Phila. Suburbs (21%)
Northeastern Pa. (14%)
Pittsburgh Area (23%)
Central/North Tier (30%)

[Based on Keystone's regional definitions, Phila. Suburbs = "Southeast", Pittsburgh Area = "Allegheny" + "Southwest", and "Central/North Tier" = Keystone's "Central" + "Northwest".]

He has been making this argument at least since 2005, and Keystone's internals show that they've been using this rough breakdown at least since early 2004.

There's a couple of problems with this.  First, while Keystone provides the counties that make up its section, to my knowledge CNN does not.  While Mr. B may infer that Pittsburgh Area = Allegheny + Southwest, that's really just an educated guess. 

For example, Keystone defines Southeast as being Bucks, Chester, Delaware, and Montgomery counties.  Is this what CNN uses?  Who knows?  It probably uses them, but then again, Berks County is also part of the Philly MSA.  What do you do with counties like Lancaster?  Does CNN count it as Central/North Tier?  Or as a Philly suburb (or as a Baltimore suburbs)?  Keystone calls it "Central," but as to whether CNN agrees, your guess is as good as mine.

One way to test this, though, is to look over the course of time and determine if F&M has a history of favoring Rs over Ds in their results systemically.  So let's look at the Presidential polling from 2004.

In 2004, Keystone did show the Specter/Hoeffel race to be less close than its competitiors, although it also had one of the two polls that showed a single-digit race.  But then again, its general 23-27 point spread was pretty much spot on (Specter won by 26).

In the Presidential race they, like most pollsters, consistently showed Kerry with a 5-6 point lead, except for one poll after the R convention showing a tie (when most contemporaneous polls showed a small Bush lead), and a pre-Abu-Ghraib poll showing Bush up by six (much like the Q poll did).

In 2006, there is some tendency to be a pro-Santorum poll, but its "final answer" -- Casey +17, ended up being the most generous poll to Casey -- and pretty spot-on for that matter.

In other words, I don't think the record shows the Keystone poll has a systemic bias.  Its used the same methodology since at least 2004, and has tended to get these things pretty close to right.  I think comparing its internals to the 2004 exit poll internals is probably comparing apples and oranges.