J Clive Matthews

Today dLiberation is coming to an end - for the time being at least. I've spent the last 24 hours trying to come up with some kind of neat final post, wrapping up all the various issues we've been covering here over the last few weeks, and laying down a final judgement on the successes and failures of the Tomorrow's Europe deliberative poll.

But, of course, it's just not that simple. I've jotted down (literally) thousands of words today - re-writing, cutting and pasting, editing and starting again from scratch countless times - only to realise that there is no short, simple conclusion here. Because almost all points that could be considered failures with the Tomorrow's Europe poll could equally apply to all other forms of representative democracy.

J Clive Matthews

For the last couple of days, winning over those sceptical of such democratic innovations as deliberative polling (assuming such innovations are valid, of course) has been my chief concern. But, with EU-centred innovations, we also still have the Eurosceptics.

I've already mentioned the dominance of eurosceptics in the online English language debate, largely due to the influence (and traffic-boosting transatlantic links to the closely-knit network of right-wing American blogs) of EU Referendum, independently run by two associates of the highly eurosceptic Bruges Group thinktank, one of whom also used to work for the UK Independence Party, before turning his back on them for being too amateurish (or so I believe).

However, due to the fleeting and superficial coverage of Tomorrow's Europe in the mainstream media - some TV coverage, the occasional short article, but nothing overly in-depth - it will be to the web that most people will look in the weeks and months to come. Amongst the online coverage now will be found EU Referendum's assertion that "what is delivered is a number of findings that are both pointless and irrelevant, except that they will be treated with undue reverence by the EU commission and its lackeys, who will cite them as evidence of what the 'citizens of Europe' think and want."

J Clive Matthews

Here we come to the fundamental problem with the EU's drive for democracy - getting true representativeness is seemingly impossible in a body as vast and diverse as the European Union. Even ensuring fair representation for the member states is well nigh impossible - let alone ensuring that smaller than national-level groups are also represented. There will always be complaints: where are the German Turks? Where are the French Basques? Where are the Russian Estonians? Where are the Afro-Carribbean British?

J Clive Matthews

The claim for the Tomorrow's Europe poll was always that it was going to be a "scientific sample" of the whole of the EU. Yet - as with the European Parliament - the forced inclusion of member states with smaller populations in such a sample instantly makes it look odd. Based on population size, a truly scientific random sample of 3,500 people from the whole of the EU should expect (on average) to contain just 2.5 Maltese - yet the Tomorrow's Europe sample, thanks to the member state weighting, contained 80.

From a PR point of view, this is understandable - ensure every member state is represented, the chance for media coverage is greatly increased. But, at the same time, it makes selling the poll as "scientific" rather harder.

A genuinely random poll of 3,500 people from the whole of the EU? Fine. One weighted to ensure the inclusion of someone from every member state? This sounds less random, more likely to contain some selection bias, and therefore less convincing.

J Clive Matthews

My concerns about the statistical representativeness of the Tomorrow's Europe poll remain, despite Professor Fishkin's response - though I accept that my complete lack of knowledge of statistical theory may well be the reason why the sample, to me, doesn't seem quite right (be it for a possible under-representation of eurosceptics or the definite over-representation of people with higher educational qualifications or from smaller EU member states).

Fishkin laid out three criteria for success: was it representative, was it deliberative, and will decision-makers listen? But these criteria leave off the single most important - will the people accept the method? Because the end result of deliberative polling must be to get the people to acknowledge that such polls genuinely do reflect what the situation would be if the people themselves were better informed and more politically engaged. Otherwise the responses will always be similar to those I highlighted from Margot Wallstrom's blog.

So, be it statistically representative or not, the key problem remains - with my apparent confusion merely highlighting the issue. Tomorrow's Europe was designed as an exercise in encouraging participation and engagement. To get people involved, simplicity and transparency is key - both of process and of results. If - after nearly seven weeks spent covering the poll in-depth and questioning key organisers - I still don't quite understand how it all works or whether it should be listened to, what chance have the public as a whole?

J Clive Matthews

With polls showing around 75% supporting a referendum on the EU reform treaty, the Conservative party repeatedly using the referendum as a fresh stick with which to beat Gordon Brown, and newspapers like the Sun and the Telegraph campaigning for the people to be allowed to have their say, in recent weeks British eurosceptics have been getting really rather excited. It has long been assumed that the UK tends towards euroscepticism, and the growing support for a referendum seemed to confirm that anti-EU types are in the majority.

So with the battle not yet won and referendum calls still reverberating around the country to apparent popular accalim, it's entirely understandable that the organiser's of Saturday's Referendum Rally in Westminster were expecting attendance in the region of several thousand. After all, if pro-foxhunting rallies in Cardiff could draw 6-10,000 a few years back, when a majority of the population were against foxhunting, surely something where the majority are in favour taking place in more easily-accessible London would do even better?

Nope. The final turnout has been estimated at somewhere in the region of just 3-500. Yep - that's three to five hundred. I'm not missing a zero, and that hyphen is not meant to be a comma.

J Clive Matthews

Two weeks after the deliberative poll, and around a month after the initial poll, we now appear to have all the results we need to assess the representativeness of the Tomorrow's Europe poll, with the release of the results of the initialal poll of 3,500 (PDF). This was the group from which the sample of 362 was taken - and so their representativeness should be compared to this initial poll. I've already noted my suspicions about the demographic representativeness - but, of course, what's needed as much as anything in an exercise like this is representativeness of opinion.

So as one of my suspicions was that pro-EU types would be more likely to take part, let's take that as a first point of comparison - especially as this is surely the most easy to measure political opinion when it comes to any aspect of EU politics. As a bonus, support for the EU is one of the things tested in the regular Eurobarometer opinion polls (most recent PDF), and so this makes for an easy point of comparison to a larger sample.

J Clive Matthews

Vice-President of the European Commission Margot Wallstrom, instigator of the Plan D programme under which Tomorrow's Europe was launched, has had a gander at the results of the poll and put up some thoughts on her blog. She picks out a few interesting results, but what is more interesting is perhaps not the areas that were intended.

First, there's the mention of other past "Plan D" projects (some of which I'd never even heard of, and I generally try to keep up with these things), and the plan to get a sample of 250 participants from all of them to go to Brussels in December and discuss how these events were run and affected them.

Second, however, is the comments the blog post has so far received.

J Clive Matthews

So, you're a multinational political party made up of 33 national parties from every member state of the EU. To gear up for the 2009 European Parliamentary elections, which will for the first time be fought by (some) parties EU-wide, rather than merely nationally, you reckon "why not get the people involved and ask them what we should campaign about?"

And so the Party of European Socialists launches a major new online consultation, designed to get up a proper debate about their policies and the future of the EU, following the reform treaty's expansion of the European Parliament's powers. "Hurrah! That's just the sort of initiative we need to give the public ways to get involved in EU decision-making!", we all cry.

J Clive Matthews

Tomorrow's Europe was part of the European Commission's "Plan D" ("for democracy, dialogue and debate"), launched in October 2005 with the aim of getting the people of Europe discussing the EU.

So, two years on, if you did a Google search for "EU debate" you'd expect this to come pretty high up, wouldn't you? People hunting for somewhere to discuss the EU would be likely, after all, to enter those terms to look for a forum for discussion.

But what actually comes top when you search for "EU debate" - with or without inverted commas? Erm... A three-year-old post by me at my personal blog's old home...

J Clive Matthews

As the Tomorrow's Europe deliberation was designed to encourage debate as much as anything, it was intriguing to see this analysis of the current state of the EU debate in the UK over at A Fistful of Euros:

"More than ever before, the entire tone of the debate about Europe in the UK seems deranged. But this time out, it also seems to be increasingly recognised that this is so.

"In a sense, the whole row has become conventionalised to a degree where it is a mere set of gestures. I recall the debates about the Euro in the late 90s and early 00s, and on the various treaties of the same period, to say nothing of the Maastricht ratification, the daddy of them all. This has had none of the same fire, despite the Sun reaching new heights of linguistic escalation and new depths of journalistic debasement."

J Clive Matthews

In the latest part of his ongoing series about the role of democracy in the EU, Paul Davies gave the following quote from Aristotle:

"a great state is not the same thing as a state with a large population. But certainly experience also shows that it is difficult and perhaps impossible for a state with too large a population to have good legal government." - Aristotle, Politics, 1326a

It's an assertion that bears much consideration - especially when combined with the language difficulties of the EU.

J Clive Matthews

My initial thinking about the Tomorrow's Europe poll was that the aim was to see what would happen if a representative selection of EU citizens were given access to better knowledge. "What's the point in that?", I thought, "as soon as you make them more knowledgable than the average, they cease to be representative, and so the final findings of the poll will be useless. All you'll prove is that people who know what they're talking about will make different choices to people who don't - which is both obvious and hardly of any use to policy-makers."

Then, on first looking at the Tomorrow's Europe briefing materials, my initial reaction was that they were packed full of differing opinions, but contained little in the way of actual facts. "How," I thought, "can you come to a considered opinion on any given issue without looking at the background information and trying to weigh up the facts? How are these people meant to increase their knowledge and make informed decisions when all they are being exposed to is subjective opinion rather than objective evidence?"

But I think I may well have been missing the point.

J Clive Matthews

One of the key arguments supporting any kind of deliberative democracy is that the process of deliberation leads to an increase in knowledge and understanding - with the natural assumption that the more they improve, the more considered the opinions.

However, as already noted, one of the questions selected by one of the groups for the final Q&A with the experts, after three days of deliberation, was "what is the role of the European Parliament within the EU institutions". With such a fundamental aspect of the way the EU runs still not understood by at least some of the participants on the final day, just how much was their knowledge actually increased?

Well, this was one of the things the poll tested (PDF).

J Clive Matthews

Such was the claim of the organisers of the Tomorrow's Europe poll, and the more I think about it, the more right I think they were.

Because, you see, the more I've been pondering whether or not the 362 people who attended the deliberation can truly be representative of the 500 million people who make up the EU, the more I've started to wonder just what the chances are of any "representative" body from such a large group looking anything like the group as a whole.

J Clive Matthews

While we wait for the results of the initial survey of 3,500 Europeans to be released, to enable comparison with the results of the surveys of the 362 participants in the Tomorrow's Europe deliberation itself, perhaps comparisons with other surveys may be of help. The most obvious, of course, is the regular Eurobarometer surveys, polling the people of Europe on a country-by-country basis - and conducted by the same polling company, TNS Sofres, as conducted the initial survey for Tomorrow's Europe.

However, while comparisons to past opinion polls of EU citizens can help us check the representativeness of the participants this time around, part of the argument for Tomorrow's Europe is that the deliberative process enables the participants to come to more informed decisions. As such, perhaps the real point of comparison should be between the participants and those who take the decisions within the EU? Here, our best option appears to be the Compagnia di San Paolo's European Elites Survey - a poll of MEPs and senior workers at the European Commission and EU Council.

J Clive Matthews

Having been spotted by a friend in the background of Friday night's Newsnight coverage of the Tomorrow's Europe poll, I naturally enough scampered off to make sure that I wasn't to be spotted picking my nose just over the presenter's shoulder (as happened a few years ago when I got rather tipsy on free champagne at the Channel Four Political Awards...). To watch the report, go here, click "watch latest programme", and scroll forward to the 22 minute mark - assuming you're doing it on Monday 22nd.

Picking one's nose is, of course, a disgusting habit. If you spot someone picking their nose, you'll doubtless think rather less of them. But what if it's only that someone tells you that they've seen someone pick their nose - something the accused denies vehemently? When you haven't witnessed it yourself, and have no hard evidence to rely on, who do you believe?

Newsnight asks, at the start of its report on Tomorrow's Europe, "will putting facts before prejudice change what they think?" But throughout the weekend, factual information on the EU was sparse, to say the least.

J Clive Matthews

Nice overview article from one of Britain's leading pro-EU writers, Timothy Garton Ash, on the events of last weekend over at the Guardian. I can't say I spotted him amongst the attendees, but there were quite a few.

Garton Ash seems to have come to similar conclusions about the poll as I am heading towards, however: "More interesting than any result is the experiment itself." The reason? Simply because this was indeed the first time that people from all 27 member states were brought together in one room and allowed to chat amongst themselves in their own languages, all simultaneously interpreted.

J Clive Matthews

To make sense of the results of the Tomorrow's Europe poll, we need:

1) The results for the 3,500 sample

2) The results for the 362 participants before deliberation

3) The results for the 362 participants after deliberation

However, so far the only results given out don't quite make clear what they actually are. This PDF of opinion changes, for example, gives before and after figures, but doesn't make clear if the "before" is the entire sample of 3,500 or merely the 362 who took part in the deliberation.

J Clive Matthews

They're now available. Analysis to follow - but any input much appreciated...

J Clive Matthews

The argument for the Tomorrow's Europe poll's representativeness (the first of the three criteria for success) hinges on the claim that it was a "scientifically representative sample" of the population of the European Union. To ensure this scientific representativeness, random sampling was chosen. (Random sampling's benefits lie in simple probability - given a large enough sample, a random selection should produce a representative cross-section of the thing being sampled.)

As such, a random sample of 3,500 people from an EU population of nearly 500 million should end up being fairly representative of the whole*, and a random sample of 400 of those 3,500 should in turn produce a representative sample of that initial sample. Hence the repeated claims by the poll's organisers of creating "the EU in microcosm", and deliberative polling's mastermind, Professor James Fishkin, arguing that "The microcosm chosen by lot embodies political equality because every citizen has an equal random chance to take part". It all sounds fine in theory.

However, the Tomorrow's Europe poll did not take a random sample of 3,500 people from the whole of the EU. Instead, the desire to ensure that all member states were represented meant that what was actually conducted were 27 separate random samples of much smaller numbers, based on the proportion of seats each member state holds in the European Parliament (EP).

J Clive Matthews

I'm still pondering this whole representativeness issue (see posts here and here), and still in need of help with the figures. A few more things have occured to me, however - largely based on my own response to any requests to take part in opinion polls.

On the occasions I've been asked, my tendency is only to take part in polls and surveys if I have respect for the organisation for which the poll is being conducted. This seems to be acknowledged as standard wisdom in some quarters.

For example, at a publishing company at which I used to work, the opinions of readers about the content of the magazines (as revealed in the occasional reader surveys, or via letters sent in) were almost always ignored - because those who went to the effort to fill out the survey or to actually write in were so unrepresentative of the readership as a whole as not to be worth listening to. Instead, it was the information about their incomes and lifestyles to which attention was paid, to provide handy statistics to potential advertisers.

When it comes to an institution as controversial as the European Union, can we really take as representative those who were willing to take part in an event which even the most idealistic pro-European must surely be aware is as much a PR exercise as an attempt at a genuine consultation? Does their willingness to participate not make them unrepresentative of itself?

J Clive Matthews

Of Professor Fishkin's three criteria for success for the Tomorrow's Europe Deliberative Poll, the first - representativeness - we are still working on (and help is still much appreciated). The second is "Is it deliberative" - and on that front it's still a bit too early to say as the results won't be released until Thursday.

However, there is enough information available to make an educated guess. By the Fishkin model, deliberation appears to be proven if the participants are shown to have changed their opinions during the course of the weekend's events. But is this enough to demonstrate that true deliberation has taken place?

J Clive Matthews

Below follows hastily-recorded partial transcripts / paraphrases (so don't take this as their exact words) of some of the comments made by Professors Fishkin and Luskin on the representativeness of the Tomorrow's Europe deliberative poll, both at the opening press conference and in conversation later - but all from before the demographics of the attendees were made available. So, do the attendance demographics justify their claims?

J Clive Matthews

Do you have number crunching skills we can call on? Right here is a spreadsheet with what demographic data is available for the Tomorrow's Europe Deliberative Poll. You can login as readers (@opendemocracy.net), password: readers . You can also download the Excel version from here.

Some of our questions:

• what are the entire EU proportions for these demographics?
• what is the confidence level that the organisers have used to determine confidence?
• are there any borderline cases?
• why have these demographic categories been chosen for control?
• how badly under-representative is the nationality sample?
• can we draw any inferences about the impact of nationality on belief given the sample size?
• what can we say, overal, about the degree of representativeness of the sample?
• what factors would you have controlled for if you had been designing the experiment?

Professor Fishkin, one of the masterminds of the deliberative polling method, has set out three criteria by which to judge the success of the Tomorrow's Europe deliberative poll. The first, and arguably most important of these is "Is it representative?"

J Clive Matthews

I've got the breakdown of the demographics of the participants, and need to see just how closely it correlates to the population of the EU as a whole - because, after all, one of the three criteria for success is to see if the sample is "representative", and pretty much the only way to check that is to look at the demographics.

However, I've got a few problems. First, I'm no statistician. Second, all they've provided is a comparison of the actual 362 participants and the initial sample of 3,500, rather than stats from the whole EU (which are notoriously tricky to get hold of). There's also no information about the urban/rural split, incomes, race or religion, which strike me as major oversights.

J Clive Matthews

Often in French, but still - from French station TF1. It should give a good indication of what it was like to have so many different nationalities all crammed together:

A general overview, including an interview with Professor Fishkin

Friday's introductory session, in multiple languages, mostly dubbed into French when not originally French

The first expert Q&A, as above - hope your French is good...

J Clive Matthews

The logistics of this last weekend were quite incredible - even more so than I had imagined. Yes, I may have been a tad critical of the pre-poll information distribution, but considering the sheer scale, it's surprising they got anything out.

Anyway, back to the main logistics. In the end there were 362 participants, each of whom had to be provided with transport, food and accommodation.

J Clive Matthews

Internet access within the European Parliament building has not been as easy to get hold of as you might think, hence the lack of posts over this weekend. Having sat in on small group discussions, spoken to various organisers and participants, and seen more about how the whole thing works, I'll hope to report in full over the next few days, having written up my extensive notes. Liveblogging, however, has sadly proved impossible...

J Clive Matthews

(Hastily scribbled from the bowels of the European Parliament building...)

The main parliament chamber is surprisingly bright - especially for anyone used to the ancient gloom of the House of Commons. Also unsurprisingly rather larger than the Commons, with 780-odd seats for MEPs.

I also have a lot more respect for MEPs now - the headphones necessary for the simultaneous interpreting (most of this event = conducted in French). Incredibly uncomfortable... No padding and far too tight, even after adjustment... To sit through lengthy political debates in any parliament can be mind-numbingly tedious - to have to do it while your head's being squashed would be a nightmare. A form of torture by multilingualism.