The difficulties of analysis

J Clive Matthews
19 October 2007
The European Parliament, Brussels

To make sense of the results of the Tomorrow's Europe poll, we need:

1) The results for the 3,500 sample

2) The results for the 362 participants before deliberation

3) The results for the 362 participants after deliberation

However, so far the only results given out don't quite make clear what they actually are. This PDF of opinion changes, for example, gives before and after figures, but doesn't make clear if the "before" is the entire sample of 3,500 or merely the 362 who took part in the deliberation.

My guess is it is of the 362 (largely thanks to the very high proportion of people who look favourably on the EU, because my guess is that people willing to take part in a poll about the EU are more likely to be loosely pro) - but as it isn't made clear what is being compared, it's hard to tell. If it is only the 362, however, then where are the figures for the 3,500?

There are so far two more sets of results - one showing that (surprise surprise!) access to information leads to increased knowledge (PDF), the other comparing the attitudes of old and new member states (PDF).

Herein, as far as I can see, lies another problem. According to this PDF, thanks to the relative population sizes (or, more accurately, thanks to the relative numbers of seats held in the European Parliament), the old member states (the fifteen members of the EU before 2004) had 257 participants, compared to the new member states' 105.

Is a sample of 105 really large enough to count as representative of the alleged groupthink of 12 countries? Where, in any case, is the justification for lumping together countries as diverse as Cyprus and Poland, Malta and Estonia?

And surely if the sample is merely 105 people, then to claim that a change in opinion of a few percentage points is significant when a 6% swing could merely indicate that just six people changed their minds, is somewhat misguided at best? Even a 10% change in opinion could, with these numbers, represent ten or eleven people in just one of the 18 small discussion groups.

In other words, without access to the raw data, it is very hard to analyse the significance of these figures. Nonetheless, I'll continue to give it a pop - and help and advice is still very much appreciated.

Had enough of ‘alternative facts’? openDemocracy is different Join the conversation: get our weekly email


We encourage anyone to comment, please consult the oD commenting guidelines if you have any questions.
Audio available Bookmark Check Language Close Comments Download Facebook Link Email Newsletter Newsletter Play Print Share Twitter Youtube Search Instagram