On scientific representation and democracy

J Clive Matthews
31 October 2007

The claim for the Tomorrow's Europe poll was always that it was going to be a "scientific sample" of the whole of the EU. Yet - as with the European Parliament - the forced inclusion of member states with smaller populations in such a sample instantly makes it look odd. Based on population size, a truly scientific random sample of 3,500 people from the whole of the EU should expect (on average) to contain just 2.5 Maltese - yet the Tomorrow's Europe sample, thanks to the member state weighting, contained 80.

From a PR point of view, this is understandable - ensure every member state is represented, the chance for media coverage is greatly increased. But, at the same time, it makes selling the poll as "scientific" rather harder.

A genuinely random poll of 3,500 people from the whole of the EU? Fine. One weighted to ensure the inclusion of someone from every member state? This sounds less random, more likely to contain some selection bias, and therefore less convincing.

400 people randomly selected from a larger group of 3,500? Fine. 3 people randomly selected from a group of 80 to represent Malta, plus 10 randomly selected from a group of 90 to represent Belgium, 15 from 105 for the Netherlands, 50 from 330 for Germany, etc. etc.? You'll have a tough time convincing anyone that the smaller samples - and therefore the final participants in the poll - have any chance of being representative. There may well be well-considered scientific methodology to support such an approach, but it's going to be far too complex for the majority of people to understand, and they will therefore most likely reject it.

Because in the end it's all about presentation. The case made by Professor Luskin for random polling as the most scientifically valid is compelling. It's participation by lottery - everyone has the same chance of being called to take part. That the man on the street can understand. But as Tomorrow's Europe's sample wasn't truly random across the whole of the EU, selling it as a lottery - and therefore scientific - is not going to convince.

Of course, lack of strict, scientific/statistical representativeness doesn't mean that the excercise was a waste of time, nor that the results of the poll should be ignored. Is the United Kingdom parliament statistically representative? Of course not. Is the US House of Representatives? Nope. Is any democratic chamber in the world? Not a chance.

But if this is the case, will any democratic system bar one run - as in Switzerland - with the repeated use of referenda of the whole population ever be able to convince the people that their concerns are truly being represented? Why do we accept democratic systems in which the people are consulted only once every few years, and yet find fault with so many alternative proposals - be they deliberative polls or citizen consultations - which are surely no more unrepresentative of the population as a whole?

Had enough of ‘alternative facts’? openDemocracy is different Join the conversation: get our weekly email


We encourage anyone to comment, please consult the oD commenting guidelines if you have any questions.
Audio available Bookmark Check Language Close Comments Download Facebook Link Email Newsletter Newsletter Play Print Share Twitter Youtube Search Instagram WhatsApp yourData