J Clive Matthews's blog

Today dLiberation is coming to an end - for the time being at least. I've spent the last 24 hours trying to come up with some kind of neat final post, wrapping up all the various issues we've been covering here over the last few weeks, and laying down a final judgement on the successes and failures of the Tomorrow's Europe deliberative poll. But, of course, it's just not that simple. I've jotted down (literally) thousands of words today - re-writing, cutting and pasting, editing and starting again from scratch countless times - only to realise that there is no short, simple conclusion here. Because almost all points that could be considered failures with the Tomorrow's Europe poll could equally apply to all other forms of representative democracy.

For the last couple of days, winning over those sceptical of such democratic innovations as deliberative polling (assuming such innovations are valid, of course) has been my chief concern. But, with EU-centred innovations, we also still have the Eurosceptics.

I've already mentioned the dominance of eurosceptics in the online English language debate, largely due to the influence (and traffic-boosting transatlantic links to the closely-knit network of right-wing American blogs) of EU Referendum, independently run by two associates of the highly eurosceptic Bruges Group thinktank, one of whom also used to work for the UK Independence Party, before turning his back on them for being too amateurish (or so I believe).

However, due to the fleeting and superficial coverage of Tomorrow's Europe in the mainstream media - some TV coverage, the occasional short article, but nothing overly in-depth - it will be to the web that most people will look in the weeks and months to come. Amongst the online coverage now will be found EU Referendum's assertion that "what is delivered is a number of findings that are both pointless and irrelevant, except that they will be treated with undue reverence by the EU commission and its lackeys, who will cite them as evidence of what the 'citizens of Europe' think and want."

Here we come to the fundamental problem with the EU's drive for democracy - getting true representativeness is seemingly impossible in a body as vast and diverse as the European Union. Even ensuring fair representation for the member states is well nigh impossible - let alone ensuring that smaller than national-level groups are also represented. There will always be complaints: where are the German Turks? Where are the French Basques? Where are the Russian Estonians? Where are the Afro-Carribbean British?

The claim for the Tomorrow's Europe poll was always that it was going to be a "scientific sample" of the whole of the EU. Yet - as with the European Parliament - the forced inclusion of member states with smaller populations in such a sample instantly makes it look odd. Based on population size, a truly scientific random sample of 3,500 people from the whole of the EU should expect (on average) to contain just 2.5 Maltese - yet the Tomorrow's Europe sample, thanks to the member state weighting, contained 80.

From a PR point of view, this is understandable - ensure every member state is represented, the chance for media coverage is greatly increased. But, at the same time, it makes selling the poll as "scientific" rather harder.

A genuinely random poll of 3,500 people from the whole of the EU? Fine. One weighted to ensure the inclusion of someone from every member state? This sounds less random, more likely to contain some selection bias, and therefore less convincing.

My concerns about the statistical representativeness of the Tomorrow's Europe poll remain, despite Professor Fishkin's response - though I accept that my complete lack of knowledge of statistical theory may well be the reason why the sample, to me, doesn't seem quite right (be it for a possible under-representation of eurosceptics or the definite over-representation of people with higher educational qualifications or from smaller EU member states).

Fishkin laid out three criteria for success: was it representative, was it deliberative, and will decision-makers listen? But these criteria leave off the single most important - will the people accept the method? Because the end result of deliberative polling must be to get the people to acknowledge that such polls genuinely do reflect what the situation would be if the people themselves were better informed and more politically engaged. Otherwise the responses will always be similar to those I highlighted from Margot Wallstrom's blog.

So, be it statistically representative or not, the key problem remains - with my apparent confusion merely highlighting the issue. Tomorrow's Europe was designed as an exercise in encouraging participation and engagement. To get people involved, simplicity and transparency is key - both of process and of results. If - after nearly seven weeks spent covering the poll in-depth and questioning key organisers - I still don't quite understand how it all works or whether it should be listened to, what chance have the public as a whole?

With polls showing around 75% supporting a referendum on the EU reform treaty, the Conservative party repeatedly using the referendum as a fresh stick with which to beat Gordon Brown, and newspapers like the Sun and the Telegraph campaigning for the people to be allowed to have their say, in recent weeks British eurosceptics have been getting really rather excited. It has long been assumed that the UK tends towards euroscepticism, and the growing support for a referendum seemed to confirm that anti-EU types are in the majority.

So with the battle not yet won and referendum calls still reverberating around the country to apparent popular accalim, it's entirely understandable that the organiser's of Saturday's Referendum Rally in Westminster were expecting attendance in the region of several thousand. After all, if pro-foxhunting rallies in Cardiff could draw 6-10,000 a few years back, when a majority of the population were against foxhunting, surely something where the majority are in favour taking place in more easily-accessible London would do even better?

Nope. The final turnout has been estimated at somewhere in the region of just 3-500. Yep - that's three to five hundred. I'm not missing a zero, and that hyphen is not meant to be a comma.

Two weeks after the deliberative poll, and around a month after the initial poll, we now appear to have all the results we need to assess the representativeness of the Tomorrow's Europe poll, with the release of the results of the initialal poll of 3,500 (PDF). This was the group from which the sample of 362 was taken - and so their representativeness should be compared to this initial poll. I've already noted my suspicions about the demographic representativeness - but, of course, what's needed as much as anything in an exercise like this is representativeness of opinion.

So as one of my suspicions was that pro-EU types would be more likely to take part, let's take that as a first point of comparison - especially as this is surely the most easy to measure political opinion when it comes to any aspect of EU politics. As a bonus, support for the EU is one of the things tested in the regular Eurobarometer opinion polls (most recent PDF), and so this makes for an easy point of comparison to a larger sample.

Vice-President of the European Commission Margot Wallstrom, instigator of the Plan D programme under which Tomorrow's Europe was launched, has had a gander at the results of the poll and put up some thoughts on her blog. She picks out a few interesting results, but what is more interesting is perhaps not the areas that were intended.

First, there's the mention of other past "Plan D" projects (some of which I'd never even heard of, and I generally try to keep up with these things), and the plan to get a sample of 250 participants from all of them to go to Brussels in December and discuss how these events were run and affected them.

Second, however, is the comments the blog post has so far received.

So, you're a multinational political party made up of 33 national parties from every member state of the EU. To gear up for the 2009 European Parliamentary elections, which will for the first time be fought by (some) parties EU-wide, rather than merely nationally, you reckon "why not get the people involved and ask them what we should campaign about?"

And so the Party of European Socialists launches a major new online consultation, designed to get up a proper debate about their policies and the future of the EU, following the reform treaty's expansion of the European Parliament's powers. "Hurrah! That's just the sort of initiative we need to give the public ways to get involved in EU decision-making!", we all cry.

Tomorrow's Europe was part of the European Commission's "Plan D" ("for democracy, dialogue and debate"), launched in October 2005 with the aim of getting the people of Europe discussing the EU.

So, two years on, if you did a Google search for "EU debate" you'd expect this to come pretty high up, wouldn't you? People hunting for somewhere to discuss the EU would be likely, after all, to enter those terms to look for a forum for discussion.

But what actually comes top when you search for "EU debate" - with or without inverted commas? Erm... A three-year-old post by me at my personal blog's old home...

As the Tomorrow's Europe deliberation was designed to encourage debate as much as anything, it was intriguing to see this analysis of the current state of the EU debate in the UK over at A Fistful of Euros:

"More than ever before, the entire tone of the debate about Europe in the UK seems deranged. But this time out, it also seems to be increasingly recognised that this is so. "In a sense, the whole row has become conventionalised to a degree where it is a mere set of gestures. I recall the debates about the Euro in the late 90s and early 00s, and on the various treaties of the same period, to say nothing of the Maastricht ratification, the daddy of them all. This has had none of the same fire, despite the Sun reaching new heights of linguistic escalation and new depths of journalistic debasement."

In the latest part of his ongoing series about the role of democracy in the EU, Paul Davies gave the following quote from Aristotle:

"a great state is not the same thing as a state with a large population. But certainly experience also shows that it is difficult and perhaps impossible for a state with too large a population to have good legal government." - Aristotle, Politics, 1326a

It's an assertion that bears much consideration - especially when combined with the language difficulties of the EU.

My initial thinking about the Tomorrow's Europe poll was that the aim was to see what would happen if a representative selection of EU citizens were given access to better knowledge. "What's the point in that?", I thought, "as soon as you make them more knowledgable than the average, they cease to be representative, and so the final findings of the poll will be useless. All you'll prove is that people who know what they're talking about will make different choices to people who don't - which is both obvious and hardly of any use to policy-makers."

Then, on first looking at the Tomorrow's Europe briefing materials, my initial reaction was that they were packed full of differing opinions, but contained little in the way of actual facts. "How," I thought, "can you come to a considered opinion on any given issue without looking at the background information and trying to weigh up the facts? How are these people meant to increase their knowledge and make informed decisions when all they are being exposed to is subjective opinion rather than objective evidence?"

But I think I may well have been missing the point.

One of the key arguments supporting any kind of deliberative democracy is that the process of deliberation leads to an increase in knowledge and understanding - with the natural assumption that the more they improve, the more considered the opinions.

However, as already noted, one of the questions selected by one of the groups for the final Q&A with the experts, after three days of deliberation, was "what is the role of the European Parliament within the EU institutions". With such a fundamental aspect of the way the EU runs still not understood by at least some of the participants on the final day, just how much was their knowledge actually increased?

Well, this was one of the things the poll tested (PDF).

Such was the claim of the organisers of the Tomorrow's Europe poll, and the more I think about it, the more right I think they were.

Because, you see, the more I've been pondering whether or not the 362 people who attended the deliberation can truly be representative of the 500 million people who make up the EU, the more I've started to wonder just what the chances are of any "representative" body from such a large group looking anything like the group as a whole.

While we wait for the results of the initial survey of 3,500 Europeans to be released, to enable comparison with the results of the surveys of the 362 participants in the Tomorrow's Europe deliberation itself, perhaps comparisons with other surveys may be of help. The most obvious, of course, is the regular Eurobarometer surveys, polling the people of Europe on a country-by-country basis - and conducted by the same polling company, TNS Sofres, as conducted the initial survey for Tomorrow's Europe.

However, while comparisons to past opinion polls of EU citizens can help us check the representativeness of the participants this time around, part of the argument for Tomorrow's Europe is that the deliberative process enables the participants to come to more informed decisions. As such, perhaps the real point of comparison should be between the participants and those who take the decisions within the EU? Here, our best option appears to be the Compagnia di San Paolo's European Elites Survey - a poll of MEPs and senior workers at the European Commission and EU Council.

Having been spotted by a friend in the background of Friday night's Newsnight coverage of the Tomorrow's Europe poll, I naturally enough scampered off to make sure that I wasn't to be spotted picking my nose just over the presenter's shoulder (as happened a few years ago when I got rather tipsy on free champagne at the Channel Four Political Awards...). To watch the report, go here, click "watch latest programme", and scroll forward to the 22 minute mark - assuming you're doing it on Monday 22nd.

Picking one's nose is, of course, a disgusting habit. If you spot someone picking their nose, you'll doubtless think rather less of them. But what if it's only that someone tells you that they've seen someone pick their nose - something the accused denies vehemently? When you haven't witnessed it yourself, and have no hard evidence to rely on, who do you believe?

Newsnight asks, at the start of its report on Tomorrow's Europe, "will putting facts before prejudice change what they think?" But throughout the weekend, factual information on the EU was sparse, to say the least.

Nice overview article from one of Britain's leading pro-EU writers, Timothy Garton Ash, on the events of last weekend over at the Guardian. I can't say I spotted him amongst the attendees, but there were quite a few.

Garton Ash seems to have come to similar conclusions about the poll as I am heading towards, however: "More interesting than any result is the experiment itself." The reason? Simply because this was indeed the first time that people from all 27 member states were brought together in one room and allowed to chat amongst themselves in their own languages, all simultaneously interpreted.

To make sense of the results of the Tomorrow's Europe poll, we need:

1) The results for the 3,500 sample

2) The results for the 362 participants before deliberation

3) The results for the 362 participants after deliberation

However, so far the only results given out don't quite make clear what they actually are. This PDF of opinion changes, for example, gives before and after figures, but doesn't make clear if the "before" is the entire sample of 3,500 or merely the 362 who took part in the deliberation.

They're now available. Analysis to follow - but any input much appreciated...