About the CBS/NYTimes/Quinnipiac poll and RealClearPolitics.com

By now you have most likely seen the results of the CBS/NYTimes/Quinnipiac poll giving President Obama a 6 point lead in Florida and Ohio, and an 11 point lead in Pennsylvania. Ed Morrissey of Hot air quickly provided the sample information(Democrat/Republican/Independent) for this poll and compared them with the exit polling from 2010 and 2008:

  • Florida: CBS/NYT 36/27/32,   2008 37/34/29, 2010 36/36/29
  • Ohio: CBS/NYT 35/27/32,  2008 39/31/30, 2010 36/37/28
  • Pennsylvania: CBS/NYT 38/32/26, 2008 44/37/18, 2010 40/37/23

Guy Benson of Townhall.com illuminates with this excellent analysis of the numbers:

 This Q-poll’s D+6 sample may be slightly exaggerated, but it’s at least in the ballpark. Barring an unexpected wave, Romney will struggle to win Pennsylvania. The other two states’ numbers, on the other hand, are preposterous. Barack Obama won the Buckeye State by 4.5 points last time, underperforming his D+8 turnout advantage in the strongest Democrat year in recent memory. Quinnipiac/NYT/CBS not only keeps that partisan edge steady at D+8 in their new poll (apparently unfazed by the R+1 electorate in Ohio in 2010), it also projects that Obama will tack two additional points onto his winning margin in that state. In Florida, the results reek even worse. Obama carried the state by three points in the big blue wipeout of 2008, thanks in part to a D+3 turnout. This poll triples Democrats’ turnout edge to nine points, and — surprise, surprise — Obama’s “lead” doubles over his ’08 percentage. All of this data should be considered against the backdrop of Gallup’s latest voter enthusiasm model, which shows Republicans much more motivated to vote in 2012 than Democrats. Ed sums things up well: “In other words, these polls are entirely predictive if one believes that Democrats will outperform their turnout models from the 2008 election in Florida and Ohio…CBS/NYT polling: New partner, same issues.”

And what about that new partner?

Hugh Hewitt interviewed the assistant director of Quinnipiac Polls, Peter Brown. Mr Brown attempted to defend this latest fiasco of a poll and didn’t do very well: 

HH: I’m joined right now by Peter Brown, assistant director of the Quinnipiac Polls, Quinnipiac in the news today along with CBS and New York Times for swing state polls, which surprised a lot of people. Peter, welcome, thanks for being on the show.

PB: My pleasure.

HH: I want to start with the models, which are creating quite a lot of controversy. In Florida, the model that Quinnipiac used gave Democrats a nine point edge in turnout. In Ohio, the sample had an eight point Democratic advantage. What’s the reasoning behind those models?

PB: Well, what is important to understand is that the way Quinnipiac and most other major polls do their sampling is we do not wait for party ID. We ask voters, or the people we interview, do they consider themselves a Democrat, a Republican, an independent or a member of a minor party. And that’s different than asking them what their party registration is. What you’re comparing it to is party registration. In other words, when someone starts as a voter, they have the opportunity of, in most states, of being a Republican, a Democrat, or a member of a minor party or unaffiliated.

HH: Okay.

PB: So what’s important to understand is what we are doing is we’re asking voters what they consider themselves when we interview them, which was in the last week.

HH: Now what I don’t understand this, so educate me on it, if Democrats only had a three point advantage in Florida in the final turnout measurement in 2008, but in your poll they have a nine point turnout advantage, why is that not a source of skepticism for people?

PB: Well, I mean, clearly there will be some people who are skeptics. This is how we’ve always done our polls. Our record is very good in terms of accuracy. Again, remember, we’re asking people what they consider themselves at the time we call them.

HH: But I don’t know how that goes to the issue, Peter, so help me. I’m not being argumentative, I really want to know. Why would guys run a poll with nine percent more Democrats than Republicans when that percentage advantage, I mean, if you’re trying to tell people how the state is going to go, I don’t think this is particularly helpful, because you’ve oversampled Democrats, right?

PB: But we didn’t set out to oversample Democrats. We did our normal, random digit dial way of calling people. And there were, these are likely voters. They had to pass a screen. Because it’s a presidential year, it’s not a particularly heavy screen.

HH: And so if, in fact, you had gotten a hundred Democrats out of a hundred respondents that answered, would you think that poll was reliable?

PB: Probably not at 100 out of 100.

HH: Okay, so if it was 75 out of 100…

PB: Well, I mean…

HH: I mean, when does it become unreliable? You know you’ve just put your foot on the slope, so I’m going to push you down it. When does it become unreliable?

PB: Like the Supreme Court and pornography, you know it when you see it.

HH: Well, a lot of us look at a nine point advantage in Florida, and we say we know that to be the polling equivalent of pornography. Why am I wrong?

PB: Because what we found when we made the actual calls is this kind of party ID.

HH: Do you expect Democrats, this is a different question, do you, Peter Brown, expect Democrats to have a nine point registration advantage when the polls close on November 6th in Florida?

PB: Well, first, you don’t mean registration.

HH: I mean, yeah, turnout.

PB: Do I think…I think it is probably unlikely.

HH: And so what value is this poll if in fact it doesn’t weight for the turnout that’s going to be approximated?

PB: Well, you’ll have to judge that. I mean, you know, our record is very good. You know, we do independent polling. We use random digit dial. We use human beings to make our calls. We call cell phones as well as land lines. We follow the protocol that is the professional standard.

HH: As we say, that might be the case, but I don’t know it’s responsive to my question. My question is, should we trust this as an accurate predictor of what will happen? You’ve already told me there…

PB: It’s an accurate predictor of what would happen is the election were today.

HH: But that’s, again, I don’t believe that, because today, Democrats wouldn’t turn out by a nine point advantage. I don’t think anyone believes today, if you held the election today, do you think Democrats would turn out nine percentage points higher than Republicans?

PB: If the election were today, yeah. What we found is obviously a large Democratic advantage.

HH: I mean, you really think that’s true? I mean, as a professional, you believe that Democrats have a nine point turnout advantage in Florida?

PB: Our record has been very good. You know, Hugh, I…

HH: That’s not responsive. It’s just a question. Do you personally, Peter, believe that Democrats enjoy a nine point turnout advantage right now?

PB: What I believe is what we found.

HH: Geez, I just, and an eight point in Ohio? I’m from Ohio. Democrats haven’t had an eight point advantage in Ohio since before the Civil War. I mean, that just never happens, but Peter, I appreciate your coming on. I’m not persuaded that Quinnipiac Polls haven’t hurt themselves today, but I appreciate your willingness to come on and talk about it.

Now if it was just one skewed poll it would be easy to dismiss, but this election cycle seems to be plagued with numerous polls over sampling Democrats and all these polls are being include in the “highly esteemed” realclearpolitcs.com average of polls. Hugh Hewitt also addresses this in his recent article:

RealClearPolitics.com was begun in 2000 and in the 12 years that have followed, it has become the gold standard for aggregators of political stories, punditry and polling data.

Almost every newscast of note refers to the “RealClearPolitics average of polls,” which the commentariat has come to rely upon as a way of smoothing out the inevitable differences between polls and the various methodologies they represent.

With success comes a problem, however, and that is the old rule of “garbage in, garbage out.”

Should “RCP,” as it is known among those who rely upon it, do something to guard its currency against devaluation?

Having built the best brand in the business, ought RCP’s editors to take some role in preserving the quality of its product?

Just as Mr Brown stated; a bad poll is like pornography, you know it when you see it. In this case it’s more like garbage, you know it when you smell it and lately something really stinks.

This entry was posted in politics and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s