Print Friendly and PDF
only search openDemocracy.net

The difficulties of analysis

Subjects:
About the author

A freelance writer and editor based in London, J Clive Matthews is Managing Editor of openDemocracy's EU and deliberative democracy blog, dLiberation.

In the real world he has co-authored two books and edited numerous others (ranging in subject-matter from movies to modern Russian politics), been acting editor on a glossy history and travel magazine, editorial consultant for a big name women's magazine, a freelance news editor for AOL UK, worked in both the House of Commons and the European Commission, and contributed to publications as diverse as Starburst and the Times Literary Supplement.

Best known as Nosemonkey online, he has been blogging about British and European politics daily for several years both at his own blog and sites like The Sharpener, General Election 2005 (now defunct), AgoraVox, France 24 and the Washington Post / Newsweek's Postglobal, as well as about movies for the BBC, and has been shortlisted for blog awards by the likes of the Guardian, Deutsche Welle International and the Weblog Awards, amongst others.

The European Parliament, Brussels

To make sense of the results of the Tomorrow's Europe poll, we need:

1) The results for the 3,500 sample

2) The results for the 362 participants before deliberation

3) The results for the 362 participants after deliberation

However, so far the only results given out don't quite make clear what they actually are. This PDF of opinion changes, for example, gives before and after figures, but doesn't make clear if the "before" is the entire sample of 3,500 or merely the 362 who took part in the deliberation.

My guess is it is of the 362 (largely thanks to the very high proportion of people who look favourably on the EU, because my guess is that people willing to take part in a poll about the EU are more likely to be loosely pro) - but as it isn't made clear what is being compared, it's hard to tell. If it is only the 362, however, then where are the figures for the 3,500?

There are so far two more sets of results - one showing that (surprise surprise!) access to information leads to increased knowledge (PDF), the other comparing the attitudes of old and new member states (PDF).

Herein, as far as I can see, lies another problem. According to this PDF, thanks to the relative population sizes (or, more accurately, thanks to the relative numbers of seats held in the European Parliament), the old member states (the fifteen members of the EU before 2004) had 257 participants, compared to the new member states' 105.

Is a sample of 105 really large enough to count as representative of the alleged groupthink of 12 countries? Where, in any case, is the justification for lumping together countries as diverse as Cyprus and Poland, Malta and Estonia?

And surely if the sample is merely 105 people, then to claim that a change in opinion of a few percentage points is significant when a 6% swing could merely indicate that just six people changed their minds, is somewhat misguided at best? Even a 10% change in opinion could, with these numbers, represent ten or eleven people in just one of the 18 small discussion groups.

In other words, without access to the raw data, it is very hard to analyse the significance of these figures. Nonetheless, I'll continue to give it a pop - and help and advice is still very much appreciated.


We encourage anyone to comment, please consult the
oD commenting guidelines if you have any questions.