Kindred Spirits

I found some kindred spirits - at least when it comes to critically looking at polls and polling data. The fine folks at HillBuzz have a post up that takes a critical look at the CBS/NYTimes poll that trumpeted a 14% lead for Senator Obama. Once the critical look was taken well....I let the author speak for herself.

Hidden at the very end of the polling data, past all the questions, is the statistical breakdown of the polling sample they used to construct this poll.

Remember what we have told you before, based on 20 years of polling: Democrats have never enjoyed more than a 4 point party ID advantage. In 2006, party breakdown was 37% Democrat, 34% Republican, and 26% Independent. In 2004, it was 37% Democrat, 37% Republican, and 26% Independent. We believe party ID this year will be 39% Democrat, 35% Republican, and 26% Independent — once more returning Democrats to a 4 point party ID advantage.

The CBS/NYT poll uses a party ID sample of: 37% Democrat, 29% Republican, and 33% Independent.

Don't take our word for it. Go out to the PDF of the CSB press release that was sent out announcing the poll. In it you will find...

This poll was conducted among a random sample of 1070 adults nationwide, including 972 registered voters, interviewed by telephone October 10-13, 2008.

Polls of "registered voters" are not as accurate as polls where the sample includes "likely voters", just take a look at your states voter turn out every election.

When the break down the questions and answers, the poll bounces answers back and forth between "registered" and "likely" voters but it never really tells you how they made that determination...at least not until you get down to the very last sentence on page twenty where they say...

*Every registered voter is included in the likely voter model, and is assigned a probability of voting, which is used to calculate the likely voter results. The sum of these probabilities is the effective number of likely voters.

It is a known fact that every registered voter is NOT a likely voter. Just take a look at voter turn out from every election from every state going back to the time when records of this kind were first compiled!

This is why I say the Hillbuzz ladies (and I assume a gent or two) are kindred spirits. In another post they reproduce this list of 20 questions that every journalist (and consumer of journalism) should ask from National Center for Public Polling. They also take it a step further and they show how a skewed poll can really distort reality.

Here are two charts: the first one shows the biased poll that shortchanges Republican party ID, and the second poll shows a more realistic party ID breakdown — note the difference: an 11-point Obama lead shrinks down to just 4-points when correct party ID breakdown is used.

As I said the other day - there could be a good reason that the pollsters are oversampling thanks to our new friends at ACORN. However, we don't know this for certain. If the sampling is being skewed due to the increased Democrat registration and it appears that pool of registrants are tainted, watch the polling and the methodology tighten up in the coming 10 days.

All I can say is the next 20 days are going to be fun to watch.