Polling to Death

NatPost reports the Tories are on the verge of winning a majority government. It's a good headline but that's all it's good for.

Darrel Bricker should be saluted, not for telling the truth, but for managing to spin the poll into a headline for the National Post. So many people really don't know how to read polls, don't know the basics of statistics, and are easily drawn to the wrong conclusions. Especially if Pollsters feed those perceptions.

This poll had a survey size of 2000. This should produce a confidence interval of somewhere between 2-3%. All a confidence interval means is that if that same poll was redone 100 times, 95 times out of 100 the numbers would be within 2-3% of the first poll you did. That is what they call a 95% Confidence Interval and is an industry standard. Now depending on the question you ask, the time of day you ask it and a host of other factors the accuracy of your poll can be affected. Merely by asking a different question you can get significantly different results.

This poll has a national confidence interval of 2-3%. However what determines if the Tories are within majority territory are the numbers in Ontario. The numbers in Ontario are based on about a 1/3rd of the survey size. That means a confidence interval of about 5%. The numbers given for Ontario was 40% for the Tories, and 35% for the Whigs. The intervals between Tory and Liberal support overlap. That is usually a pretty good indication that there is a high statically possibility that those two proportions could be equal. In other words: the Tories and the Liberals might actually be tied in Ontario. In fact other polls coming out lately have shown much closer numbers for Ontario would you believe.

So seeing this Darrel Bricker, if he wanted to be statically sound, would have said nothing of the sort that he did. He could have said there is a slight possibility of a Tory majority. If he wanted to say more he should have done a poll just of Ontario - because as most agree a difference of 5% could make a majority or minority at this point. But coming to conclusions like this one - no matter how satisfying to myself personally - makes no sense.

I would feel dishonest if I said nothing about it.