Monday, 7 September 2009

Media oversight from the unexpected angle.

Much has been written about how the Daily Show with Jon Stewart can often be the hardest hitting political show on the American networks despite being produced by and aired on a cable comedy channel. In terms of television output luminaries such as Marcus Brigstocke and Iain Lee & Daisy Donovan have tried to mine the vein of satirical current affairs but none of them have worried Jeremy Paxman and television satire in this country remains mostly the preserve of impressionists and panel games.

There is one area in this country in which a programme you might not expect is doing a fine job at holding people to account. More or Less goes out on Radio 4 after the World at One on a Friday lunchtime and is a co-production with the Open University. This seemingly unassuming half hour of statistics can be (glibly) described as doing for numbers what Ben Goldacre does for medicine. While not the first important figure they have picked apart (there is a very good piece about how many CCTV cameras there are in the UK for example) they hit the geek headlines over the weekend over "7m illegal file sharers". After first finding out that the Strategic Advisory Board for Intellectual Property was quoting a figure that came out of unpublished research commissioned for the BPI which had been laundered through another report from the same research outfit via UCL they then examined the methodology and numbers from which it was derived which brought both the small sample size and seemingly overstated overall population figure under scrutiny.

The programme is also blessed in that it has its archive online going back to 2003 so the good work can be cited.

3 comments:

David Gerard said...

The sample size (about 1000) is not an issue - remember that the quality of the result (the resolution) depends *entirely* on the sample size, *not* on the size of the population. Doesn't matter if the population is ten thousand, 100,000, a million or fifty million - if the sample size is 1000, you'll get a +/- 10% figure, which is plenty good enough to go on with.

(This is a surprising mathematical result, which is why first-year statistics students get it beaten into their heads: RESOLUTION relates only to SAMPLE SIZE, it does NOT relate to POPULATION SIZE.)

The problematic aspect is the massaging they then did to the figure.

Tony Kennick said...

David, isn't the number they used for the online population here being larger than other reported figures important because they are reporting a number not a percentage?

David Gerard said...

Yes, that's part of the highly questionable fudging. I'm just addressing the bit about "small sample size."