Tuesday, April 8, 2014

To All Who Bash a Certain Pollster...

Final result in Quebec (April 7): 41.5% PLQ, 25.4% PQ, 23.1% CAQ, 7.6% QS

Final Ipsos Poll (April 1): 40% PLQ, 28% PQ, 18% CAQ, 12% QS, avg. 3.4% off
Final Leger Poll (April 3): 38% PLQ, 29% PQ, 23% CAQ. 9% QS, avg. 2.2% off
Final Angus Reid Poll (April 4): 39% PLQ, 27% PQ, 25% CAQ, 7% QS, avg. 1.6% off
Final EKOS Poll (April 3): 40% PLQ, 27% PQ. 21% CAQ, 9% QS, avg. 1.6% off
Final Forum Poll (April 3): 44% PLQ, 24% PQ, 23% CAQ, 6% QS, avg. 1.4% off

Yeah, you can stop now.


  1. I don't know if forum is particularly unreliable or not, but to make a claim based on one poll result is poor reasoning, and unlikely to convince anyone. Try something like pull the last N, where N >> 1, provincial elections where fourum polled and compare their performance to other pollsters.

    1. You're right, though I'm not trying to make any kind of formal point. Forum does no worse or better than the other pollsters, though if you ask anyone else, they're the worst pollster in Canada. That is my general point.

    2. To make it more clear, in the last Quebec election, Forum was not as far off as any of the other pollsters, but because they gave the PQ more support and everyone hates the PQ and we had the result we had in 2012, people decided to bash Forum. Same goes for BC and Alberta, where Forum performed no worse than the other pollsters who ALL got it wrong, yet Forum takes the brunt of the attack against pollsters.

    3. What really bugged me when it came to Forum-bashing was all the brunt they got from other pollsters after their polls in Brandon-Souris, when NOBODY ELSE even polled the riding.

      It was easy for every other pollster to criticize since they didn't even poll themselves...

  2. This election was one that was remarkably well called by the polls (in contrast to say the last BC election). In an election where all the pollsters cluster near the right answer, happening to be the best of those isn't really meaningful. The real test of different polling methodologies and voter models comes in the really divergent elections.

    But sure, those that claim Forum is worth less than toilet paper are obviously being silly

  3. A decade ago, it was Ekos known for being the "worst" because they were the ones who did most of those polls showing the NDP ahead back in 1987, and the Reform/Alliance doing rather well, etc; they became known as the firm that overstated whatever trend was hot at the time.

    Prior to that Gallup had pie on it's face. Between Ekos and Forum, SES was the "bad firm". Ekos survived, SES became Nanos, and Gallup no longer polls in Canada.

  4. How about the regional breakdown? how well did they get that?

    1. I thought about checking that but the fact is that the sample sizes for regional breakdowns are just not that useful to begin with - they're more for eye candy than anything else. But taking a quick look at the big three (Montreal RMR, Quebec RMR, and the Regions) they were not too far off.

    2. Thanks Kyle. Do you have the actual results in those regions? I'm not sure where to find this info.

    3. I have them on hand but ThreeHundredEight.com has the info as well, here's a direct link to the bit that has it: http://1.bp.blogspot.com/-64x8jLjlql4/U0QaZBaR4RI/AAAAAAAASTs/XIcFd2bhtO0/s1600/Post+Regions.png

    4. 308 may not be correct.
      Compare, for example, the Greater Montreal area http://library.concordia.ca/research/subjects/geography/Cadastre_Map.jpg
      To the 450 area code http://www.cnac.ca/area_code_maps/southern_quebec_area_code_map_highres.png

      Ekos polls by area code. A lot of CAQ and near-CAQ ridings are in the 450 but not in the greater Montreal area.

      Keep that in mind for their Ontario numbers. Their GTA is likely the entire 905
      which excludes Barrie but includes St. Catharines, and goes almost all the way to Belleville.

      This also explains why the poll made a reference to north-west ontario