Yet more on representativeness

J Clive Matthews
17 October 2007
No, that isn't Sir Menzies Campbell looking at a piece of paper in the middle there... I think...

I'm still pondering this whole representativeness issue (see posts here and here), and still in need of help with the figures. A few more things have occured to me, however - largely based on my own response to any requests to take part in opinion polls.

On the occasions I've been asked, my tendency is only to take part in polls and surveys if I have respect for the organisation for which the poll is being conducted. This seems to be acknowledged as standard wisdom in some quarters.

For example, at a publishing company at which I used to work, the opinions of readers about the content of the magazines (as revealed in the occasional reader surveys, or via letters sent in) were almost always ignored - because those who went to the effort to fill out the survey or to actually write in were so unrepresentative of the readership as a whole as not to be worth listening to. Instead, it was the information about their incomes and lifestyles to which attention was paid, to provide handy statistics to potential advertisers.

When it comes to an institution as controversial as the European Union, can we really take as representative those who were willing to take part in an event which even the most idealistic pro-European must surely be aware is as much a PR exercise as an attempt at a genuine consultation? Does their willingness to participate not make them unrepresentative of itself?

A post on Tomorrow's Europe from Ron Lubensky, a research assistant at the University of Sydney's Department of Government and International Relations who's looking into deliberative methods, heightened this feeling (selectively quoted with permission):

"The organiser delegated a polling company to randomly select 3500 citizens from across the EU for the initial survey. Let's assume that is a truly random and stratified sample that reflects the demography of each country.

"But how were the 400 attendees to the DP selected from those 3500? The media release states that they are selected scientifically. Of course, they weren't conscripted, so they would have had to self-nominate from the sample. Are they still a microcosm? No, they are not. They would be just the ones who were confident and affluent enough to go to Brussels for a long weekend. Most importantly, they would have been the ones who believed in the whole process.

"...A large chunk of people believe that deliberative democracy is elitist and a leftist plot against their civil liberties. If they were invited to contribute to a Deliberative Poll, they'd laugh it off as a waste of time. They wouldn't participate, but then they'd tear it down because they don't believe that it's representative!!"

Let us take, to prove the point in an utterly unscientific way, a comparison of participants from three member states of equivalent sizes (all with 78 seats in the European Parliament).

In the initial sample of 3,500, there were 300 French participants, 301 Italians and 302 British. In the final sample of 362? 41 French but only 28 Italians and 28 British. France famously had a very heated and detailed nationwide debate in the run-up to the 2005 referendum on the EU constitution, widely praised for being one of the best mass debates on the EU in the organisation's history.

Considering that all participants were to be reimbursed for travel and expenses, as well as offered "thank you" payments for attending, does the heightened response rate from French invitees indicate a greater interest in EU matters in France as a whole - and if so does this make it less representative?

Do the additional 13 French attendees (13 being more than the participants for Austria (11), Belgium (11), Bulgaria (9), Cyprus (3), the Czech Republic (12), Denmark (9), Estonia (3), Greece (11), Finland (8), Hungary (11), Ireland (6), Lithuania (4), Luxembourg (2), Latvia (7), Malta (3), Portugal (11), Sweden (9), Slovenia (4) and Slovakia (7)) mean that the poll was disproportionately swung in favour of French opinion?

And, more generally, is this kind of random selection really capable of being representative when a) it's for a one-off poll, so the law of averages that normally applies to random selections does not apply, and b) it relies upon being able to persuade those who have been randomly selected to actually participate?

Had enough of ‘alternative facts’? openDemocracy is different Join the conversation: get our weekly email


We encourage anyone to comment, please consult the oD commenting guidelines if you have any questions.
Audio available Bookmark Check Language Close Comments Download Facebook Link Email Newsletter Newsletter Play Print Share Twitter Youtube Search Instagram