Wednesday, 1 August 2007

Our survey says

"All single people are that way because they are choosy" - 100% of respondents in our poll who described themselves as single declared they were choosy

"More than three-quarters of the population prefer Chips to Daddy" - In a shock poll yesterday fathers lost out to fried potato products.

"Couples yearn to be free spirits and have a good time while singletons wish for more stability" - 65.38% of people in a relationship ticked the free spirit box while 76.92% wanted a 'good time' the want for stability was near universal for the singles.


All of those headlines are of course mine and I will happily admit that the poll itself is utterly unscientific, it was self selecting, has a very small sample size and was conducted on a social networking website. It was of course part of illustrating a point, can we trust polls? I am not, despite using the standard newspaper model of sensationally cherry picking results, questioning can we trust reporting on polls. Of course we can't; they are even more a vehicle for putting forward that particular paper's political philosophy than the comment pages with the added gravitas of numerical facts. So even when they aren't sliding into a reporting error of Guardian proportions you have to assume there is Monty like spin going on from all sides.
No it is the underling polls that I am still suspicious of, what questions have the asked around the questions being highlighted, who exactly did they ask etc. The big players: ICM, YouGov, MORI have started putting the whole of the poll online but still there are hidden darknesses about weighting and the issue that to go looking for that, you need to know who conducted the poll in the firstplace (or go looking for it on all their sites).
What do I want? Any use of poll data should be required to have a citation (to be honest any statement of a fact in an ideal world would have one, but newspapers haven't quite got hyperlinking from paper sorted yet) to the full data including all questions and details about how any weighting is done (which demographics were adjustments made for, for example).
Of course, I still won't believe polling numbers, they are after all just statistics.

No comments: