Monday, March 28, 2011

Quick Quebec Quackery

I noticed that CROP recently had a poll out that showed the Liberals at 11% in Quebec, and the NDP at 20% - which would apparently give the Libs 3 seats and the NDP 5, the rest split up among Cons and Bloc Quebecois, according to's Eric Grenier.

This is, obviously, an interesting poll, if only because it has such wonky, and terrifying, numbers. But I've said before that CROP, along with another online Quebec pollster called Leger, has been drifting increasingly towards both the crazy, leading me to suspect that this may in fact be another voodoo poll, as are a lot of online pollsters.

To demonstrate the difference between traditional pollsters (those that phone) and online pollsters, here's a fun graphical chart:

This chart is an average of 17 traditional polls and 11 online polls that have been released since the beginning of 2011. There is an obvious difference to be found - can you spot it?

While the Liberals don't move much (though more than the Bloc), the huge difference between the NDP vote is a figure worth noting. That's over most margins of errors in Quebec polls, and represents an obvious bias that we can't ignore when looking at a poll like CROP or Leger. It begs several questions, which I won't answer here. But, that's my piece on this issue - make your own decisions. I just offer up the data.


  1. This comment has been removed by the author.

  2. Wouldn't an online poll skew more toward the younger, educated and wealthier Canadians who can afford access to, and be motivated to learn and use, the internet? Doesn't this demographic typically support the centre or the left?

    It seems that a combination of phone and internet sampling would produce a better result. Which is the reason I think that 308 dot com is pretty darn good - an aggregate of all the polls - improving both the quantity and quality of the sample.

  3. I agree, is excellent - but there's a bias here that makes me believe that, if you can't discount the polls, they do need to have some sort of penalty against them, a lesser weight, for instance, or some way of correcting the issue.

    The first point is what many suspect to be true - though it's not necessarily true. I've read several articles that have said that online pollsters have a tendency to weigh their results a lot differently than traditional pollsters, and that their base unweighted results actually aren't very different from what you get off of phoning. That's just one issue, however.

    There's also the issue of the fact that online pollsters aren't really disrupting Conservative numbers, but rather Liberal and NDP numbers, as well as it seems Green numbers. You see the same issue in the UK and other places with our troika of major parties kind of set up. The secondary opposition party is usually penalized versus the third party, which usually gains.

    While demographics can explain it, it doesn't seem to fit the entire profile, which makes one wonder.

  4. What BS.

    As was noted above, the only relevant changes in your comparison are between the never was and the wishing they were.

    How's the vote for the never was? Did they crack 5% ever?

    Which of your two graphs have a basis in historical reality and which doesn't?

    Unbelievable, using a misinformed bias regarding online polling to attempt to smear a single poll because it makes the Libs look like the Greens.

    Man, you and Tribe, neanderthals regarding polling. Even Steve V readily admits that the online polling was the most accurate in the last few elections.

  5. Lance,

    You don't see an obvious bias there, Lance or Iance or whatever? This is just the raw data that anyone can pick up and find for themselves, and you're telling me there isn't a bit of a difference to question there? No?

    And what a croc - who have been the most accurate pollsters in the last three elections? Ekos and Nanos in 2008; Nanos in 2006; and *maybe* Leger in 2004, though half a month before the election, which still wasn't that accurate.

    Now, you want to tell me again that these online polls were accurate? Or do I have to start pulling out American, British and other data to pin facts down for you?

  6. Furthermore, "Lance," I never questioned the accuracy - I questioned the fact that there is a pretty large bias that is easily demonstrated. The fact that I can do so by just eyeballing data is a bad sign.