Wednesday, April 8, 2015

1 Alberta Votes, Google Surveys River-Sampling, and the Weird World of #ABVote Polling

The weird world of Alberta polling just got weirder.

While I sit here waiting for ThinkHQ to release its official numbers for the recent poll it put out showing the NDP second, another poll popped up on my feed showing the NDP first in Alberta, against all possible odds.

This poll comes from the folks at 1 Alberta Vote (1ABV) which are, according to their website, a group of "...citizens who are concerned about the governance of this wonderful province and are tired of the scandals, fiscal mismanagement and lack of vision" of the governing Progressive Conservatives, and have thrown their support behind the initiative of Liberal MLA Laurie Blakeman to get multiple party endorsements for her candidacy. So there is an inherent bias right off the bat here, but that doesn't necessarily mean anything as this is not, say, a sample of 1ABV's viewers - I think. Let me explain.

1ABV has commissioned Google Consumer Surveys, which does not find its samples in the way traditional telephone, IRV or online polls do, which is to essentially ask respondents for their demographic information and continue from there. Google Surveys uses a river sampling method through online and mobile portals to find its samples, which is not too different from how, say, EKOS or Angus Reid conducts its online polls (with some big exceptions), and claims that this methodology gives it more accurate results than its competitors.

So, what is river-sampling? Essentially - this is all how I understand it, and I could be clueless, please keep that in mind - Google Surveys uses various methods to attract potential respondents to complete its polls, usually through a reward system (complete this survey, enter to win $5!), and will apportion them particular surveys based on various data sets collected through the use of IPs, web caches, browser histories, cookies, and so on. 

For example, you would click on the link to complete a survey, and Google Surveys would use the information provided by your browser and IP to give you a relevant poll for your location and inferred demographic - if you're an older person in Alberta, Google should be able to infer that through your search history and so on, and thus will give you one of its Alberta political landscape polls.

There is nothing inherently wrong with this type of polling, and it is not woo - that is the good news. It was used in the 2012 US Presidential election with, despite their claims, fairly average results, with about the same accuracy as Ipsos, Angus Reid, and most other internet pollsters and some live caller pollsters as well. It was also apparently used in the recent Winnipeg mayoral race by the successful Bowman campaign, but we don't have the numbers for those.

The bad news is that, while not woo, it is subject to the same biases that other internet pollsters seemingly have in Canada, but one it does admittedly work to correct. One of the larger issues among online pollsters is a sample size that will generally be more skewed towards those online - usually that is the younger crowd, which tend to be more left-leaning and in some cases more self-selecting (though all polls, online or live callers or IRV, are self-selective to some degree). Google Survey's hope is that by using river-sampling, it will be able to get a better and more representative sample than traditional pollsters, who generally ask for sample information from respondents themselves. Check out this white paper if you're really interested in the details.

I'm not sure if they're more successful than the traditional methods, but they're definitely on par with them, and are used by companies like Pew Research for quick reaction polling, as well as private industry and, it seems, political campaigns.

This is the first time I've heard of it being used in Canada openly, though some of our pollsters could be using similar methodologies already, I don't know. It does, however, seem like it is a legitimate way to sample and survey voters - and thus, I will be adding its results to my rolling projection, at least until I can otherwise determine if its biased or otherwise unusable (too small of a sample size, etc). But with other polls showing results not too dissimilar, I'm not sure I will.

However, I would like to point out one sentence on 1ABV's page that really annoys me:

A note on skew and polling methodology: It is likely that Google Surveys may have slight left skew in Alberta. We have noted that IVR polls in this province skew right. So, in triangulating polling data, the truth is somewhere in between.

This right here is what you should not do. Never assume a bias just because you can, and then "triangulate polling data" (which could mean anything) and say that the truth is about there. That isn't really how polling works, and leads to faulty assumptions on everyone's part. Just don't.

No comments:

Post a Comment