Sunday, March 20, 2011

The CARP Poll is a Voodoo Poll

There's been this little poll circulating the blogosphere recently done by the Canadian Association of Retired Persons (CARP) on the voting intentions of their specific voter subset - people over the age of 55. This poll essentially states that among CARP's membership, the Conservatives have had a huge drop in support, going from a high of about 52% in early February, to 41% now - while the Liberals jumped from 32% to 42% in the same time. This is taken as an indication that the Conservatives are facing something of a backlash amongst older voters.

Before anyone gets excited though, remember that a single poll is not a trend - and so far, no poll has shown the kind of drop amongst older voters that CARP has. Right away, the numbers presented must be taken with a very large grain of salt.

The other fact throwing doubt upon this poll is the fact that it's amongst CARP's own membership, and it's methodology and samples are questionable. Let me explain.

CARP's poll is an online poll of approx. 2,000 participants, all of whom are subscribers and readers of CARP's newsletter that obviously come online. That, however, is as much as we know; there is no explanation of their methodology, of any weighting system used, of regional data, voting subsets (income, race, age, gender, etc.), or whatever else you can think of. Essentially, all CARP's poll could mean is that there are a lot more older people from the GTA who are possibly lower income reading the online newsletter right now.

Which is what may have actually happened. Looking over the "raw data" of CARP's poll, you can see that 61.2% of the 2,515 votes came from Ontario, 7.9% came from Alberta, and 2.9% came from Quebec. That right there is a serious indication that your samples are flawed in a very big way.

Going on, you continue to see other big flaws in the samples, including in gender (I didn't know over 2/3 of Canadians were male), income, and even age brackets. And with no indication that the samples were weighed to battle against these results, there is a big question of validity in this poll. It's more or less a voodoo poll, something that is pretty to look at but we can't possibly base any data or hopes off of.

What really kills me is CARP's insistence that this poll is just as credible as others. It isn't. Don't believe it. Whatever these results may say, they can't be trusted. That's simply the truth. There's not likely such a big drop in Conservative support amongst this 55+ age bracket. And in no way have I seen CARP reflect any polling trend yet.

No Conservative implosion. No Liberal upswing. I hate to burst people's bubble, but this poll is not trustworthy. Simple as that - until anyone can prove me wrong, however.


  1. I don't believe we should discount that poll, the graph explains why it may be relevant. That said, I agree people running with this unique methodology, as though gospel or truly indicative of a real shift, are a bit foolish at this point. I think the problem, people like to manipulate whatever suits their predetermined bias, while simultaneously ignoring everything else.

    I would say CARP is interesting, it does denote a change, but we need to see some follow up elsewhere before people should say anything definitive.

  2. It's definitely interesting, but I still don't think its indicative of anything, simply because the samples are so biased.

    But you're right, it could still denote something, though what, who knows. We'll all have to keep an eye on the trends though.

  3. I dunno -- even if the absolute values might be overstated (and CARP itself does have a methodology section which concedes / explains that the respondents are more likely to be males, in their late 50s, and affluent, than the whole of their membership), it's the trendline that's sig: a drop of one fifth in support, among a demographic in which half the respondents usually intend to vote Con.

    And as the Star's editor points out, "The results were analyzed by a professional pollster, John Corbett, formerly of Gallup Canada."

  4. I don't trust 'em still, and "Gallup Canada" was a joke to begin with (really it was).

    It's just clear that these samples aren't credible in the least. That's concerning and throws around a lot of doubt over this poll. It could be that there is some sort of trendline - but it's yet to be repeated elsewhere.

  5. All I know for sure is that ignorance is a terrible abdication of responsibility and accountability:

    And just in case you think that I am just full of hot air, I PROVIDE THE PROOF HERE.

    I would love to hear from you.

  6. CARP is a front group for the Raging Grannies. ;)