The day before the Tomorrow's Europe deliberative poll kicks off, materminds Professor James Fishkin and Professor Robert Luskin clarify deliberative polling's claims to representativeness:
To help establish that the sample is indeed representative, the characteristics and pre-deliberation views and knowledge of the participants (the initial interviewees who attend the deliberations) are compared with those of the non-participants (the initial interviewees who do not attend).
Then, to gauge the deliberation's effect, the participants' post-deliberation views and knowledge are compared with their pre-deliberation views and knowledge. To see whether post-deliberation opinion inclines one way or the other, it may also be useful to compare the participants' post-deliberation views to the midpoint of the scale (neutrality).
For each of the comparisons, the sample should be large enough so that an important difference - whether of participants from nonparticipants, of participants after deliberating from participants before deliberating, or of participants after deliberating from neutrality - is "statistically significant" (can be confidently said to be nonzero in the population). All things being equal, larger samples make for more precise estimates (smaller "margins of error") and thus for greater statistical significance.
Two under-appreciated points need emphasis, however.
First, since the margin of error varies inversely with the square root of the sample size, the larger the sample already is, the smaller the difference a given increment will make. A sample of 200 is much better than a sample of 100, but the difference between a sample of say 400 and one of 500 is far smaller.
Second, the precision of the estimates depends only trivially on the gap between sample size and population size. A sample of, say, 400 drawn from a population of 300 million is only imperceptibly inferior to one of 400 drawn from a population of 300 thousand.
While there is no bright line, our past experience suggests that a sample of a few hundred is enough to assess the representativeness of the sample of participants and to estimate their changes of opinion. Important differences and changes are, with only rare exceptions, statistically significant.
