Nothing is necessarily as you thought it was, and you should never believe what you're told until you've had a chance to study it for yourselves
Nothing is necessarily as you thought it was, and you should never believe what you're told until you've had a chance to study it for yourselves
NavigationThe World
|
![]() |
Fishkin and Luskin reply to LupiaElsewhere on openDemocracy
The full, unedited text of the response of Professors Fishkin and Luskin to the criticisms of Professor Lupia: Skip Lupia has staked a good part of his career on the view that, by and large, ordinary citizens successfully use cognitive short cuts and simple cues to reach the same policy views and electoral choices they would reach if they knew and thought a lot more about them--and that deliberation should therefore make little difference. The evidence from Deliberative Polling challenges a great deal of his past work. Lupia’s bald assertion that there is "broad and deep skepticism about this endeavor within the social sciences" is strikingly at odds with the large number of distinguished scholars who participated in one way or another in mounting or helping to analyze Deliberative Polls. We refer readers to the projects linked on the Center for Deliberative Democracy (CDD) web site (http://cdd.stanford.edu). These collaborators, moreover, are a tiny fraction of the serious, well known scholars who have expressed interest in and appreciation of the enterprise. No doubt Skip Lupia is deeply skeptical. No doubt some of the people he talks with are too—the way the biases in selecting conversational partners and topics work, no doubt a disproportionate share of them are. But, we have so far received only one anonymous peer review from a refereed journal, out of something like a couple of dozen that has had anything remotely of the flavor of Skip’s remarks here. The claims in our Deliberative Polling research are supported by at least 21 scientific papers, some already published and the rest presented at scientific conferences and on the CDD web site. These papers are of course subject to peer review as they go through the publication process. At the moment, together with varying collaborators, we have five papers with revise-and-resubmit verdicts from top political science journals. They would not have got this far if most of the reviewers shared Lupia’s attitude. The debate in Legal Affairs was essayistic rather than data-based. Posner took the position that the public is so incompetent that deliberation would be futile. Lupia took the position that the public is so competent that deliberation is unnecessary. Neither adduced any data. It is true that studies of “deliberation” have produced some variety of results—unsurprisingly, given the considerable variety in definitions and operationalizations underlying them. Much depends on whether a deliberation is part and parcel of a decision making process (as in many committee meetings) or a prelude to one (as in an election), whether it is consensus-seeking or not, to what extent it is balanced, etc. We have tried to be clear about the kind of deliberation we are trying to capture, which is a prelude to decision-making, non-consensus-seeking, and balanced, among other characteristics. Jury deliberations, the bedrock of the literature suggesting that deliberation homogenizes opinions within groups, polarizes them across groups, and allows the privileged disproportionate influence, are very different. The participants are generally less diverse, there is much less assurance of balance, the process is decision-making, and it is designed to achieve consensus if possible. All we should claim is that the kind of deliberation we aim to capture (a) is relevant for what we might hope for in a mass public that learned, thought, and talked more about the issues before expressing its views in polls and casting its ballots and (b) seems to have effects of the sort we describe. Jury deliberations are interesting and important in their own right but far less relevant for the study of mass politics. Contrary to Lupia, a number of other scholars have independently analyzed some of our data, writing scholarly papers and, in one case, a book from them. They include Don Green, Cynthia Farrar and David Nickerson of Yale, Kasper Moller Hansen o the University of Copenhagen and Vibeke Normann Anderson of the University of Southern Denmark, Pierangelo Isernia of the University of Siena, Luigio Bobbio of the University of Torino, Pam Ryan of Issues Deliberation Australia and Roger Jowell and Alison Park of the National Center for Social Research in London and Patrick Sturgis of the University of Surrey. We have admittedly been slow about making our data more widely available. We have been the victims of our own success in entrepreneuring a continuing stream of Deliberative Polls, which has retarded our efforts to write up the results for scholarly publication. As noted, however, five manuscripts are currently at the revise and resubmit stage, and as these and other manuscripts are accepted, more data sets will be made available. We have put countless hours into creating and implementing these projects and simply want to have first crack at the data from them. When Lupia raised the issue of access to Deliberative Polling data at a recent symposium at Stanford, we replied that we would be happy to send him some. We never got a request but should still be happy to do so. Finally, It is worth noting that we are at this moment (September 27-29) conducting a "Deliberative Polling Training Institute" at Stanford to train researchers from 16 universities around the country to conduct their own Deliberative Polls, producing their own data and, we hope and expect, leading to their own independent social science publications, testing our and many other claims. James Fishkin and Robert C. Luskin In my original post, I raised questions about the transparency and effectiveness of deliberative mechanisms such as Deliberative Polls (DP). Jim Fishkin and Professor Luskin have made several claims in their response that deserve additional attention. (This version of my response is extended to parallel the complete entry of Fishkin and Luskin.)
In sum, deliberation can sometimes do great things, it can sometimes cause huge problems, and it can often have no lasting effect at all. Everyone who has a stake in Tomorrow’s Europe has a right to ask direct questions about what a Deliberative Poll can -- and cannot -- accomplish.
1.
Jim and Professor Luskin begin by suggesting that my motive in questioning Deliberative Polling is that it challenges my past work.
It is odd for an advocate of deliberation to defend his claims by questioning others’ motives. It is well known within the deliberative democracy community, that this kind of maneuver does more to impede successful deliberation rather than facilitate open conversation.
To deal with the facts of his claim, I have published a number of articles and books focuses on the conditions under which citizens can successfully use various pieces of information to make quality decisions. I show that sometimes citizens can be successful when skeptics and pundits do not expect it. In other cases, citizens are not at successful.
This is all very well documented -- as Jim Fishkin knows.
I invite you to have a look at my website (http://www.umich.edu/~lupia) and draw your own conclusion.
2.
I was invited by the editors of this blog, to offer greater clarity in what deliberative democracy can and cannot do. To this end, the main conclusion of an expansive set of scientific research (almost none of which is by me) is that if you want to propose credible means for helping citizens be more effective, it is important to understand the conditions under which citizens can -- and cannot -- use information effectively. The questions I and others have raised about deliberation come from a disconnect between claims about the effects of deliberation (such as some made by Jim) and heavily researched findings about learning and decision making that emerge from experiments and related scientific work in fields such as psychology, sociology, political science, and the neurosciences.
In this case, the science-Fishkin disconnect does not automatically imply that Jim is wrong when he makes claims about DPs, but it does mean that there is a legitimate basis for asking whether he is right.
3.
In science, a common way to handle such matters is to facilitate replication. This often entails sharing data from which key claims are made.
Later in their response, Fishkin and Luskin admit that they have "been slow about making our data more widely available." Slow is right. Their website refers to dozens of studies costing many millions of dollars including some that are over a decade old. But even as of today, there does not appear to be a single dataset that is publicly available.
They then claim that, "We have been the victims of our own success in entrepreneuring a continuing stream of Deliberative Polls, which has retarded our efforts to write up the results for scholarly publication." This response does not hold water in the scientific community. That they have been "too busy" over the last decade to post data on which they seek to base scientifically validated claims is not a credible response. Today, it takes only a few minutes to post a dataset on the web. If Jim is serious about obtaining the legitimacy for DPs that comes from transparency, then he should make his data available without delay.
While defending himself, Jim cites instances where he has shared his data. However, most of these people are **his coauthors.** This is not of much help in facilitating independent evaluations.
When people want to have a large social impact and claim scientific credibility, transparency typically means making data available for everyone.
Post new comment |
![]() |
ElectionsMost discussed articles... |
Arthur Lupia said:
Wed, 2007-10-03 15:31