On one level, today's referendum closes the political cycle that started with the eruption of the expenses scandal in 2009. That scandal brought forth a set of demands for political reform which found outlets in different channels.

On the one hand, there was limited institutional reform in Westminster itself (the creation of IPSA and the Wright Committee reforms, in particular), coupled with the deselection, resignation and even prosecution of MPs caught up in the scandal. This was mirrored at the grassroots by a rebirth of interest amongst politically engaged young people in democratic reform groups, like the Purple People who gathered in Smith Square to put pressure on Nick Clegg during the Coalition negotiations. But it was not more widely mirrored in the public, whose revulsion at the expenses affair did not translate into concerted political pressure for change - a fact which lies at the heart of the failure of the Yes to AV campaign's messages to connect with the electorate.

Within the political parties, there were attempts to harness the popular outrage about expenses to different political objectives, both substantive and tactical. Labour sought to revalorise its constitutional and democratic reform project, which had stalled since Gordon Brown's initial statements as prime minister in 2007; the Liberal Democrats used it to press for long-held reform commitments; while Cameron ruthlessly used the crisis to outmanouevre both the Conservative old guard and the Labour government, promising little by way of substantive reform but achieving a lot by way of political positioning.

The AV referendum became a fulcrum point in this set of processes when Gordon Brown announced his commitment to it to a surprised Labour Party conference in 2009. (It was something he had spent the summer thinking seriously about, and finally decided to announce it on the day itself, with some pushing from pro-reformers in the Number 10 team.) That set off a chain of events which led to the ultimately unsuccessful attempt to amend the constitutional reform legislation which was then passing through the Commons to allow for a referendum at some time before October 2011. For reasons that now rebound on Nick Clegg, the Liberal Democrats argued that Gordon Brown's unpopularity was such that if he were Labour leader and prime minister at the time of the referendum it would fail, and thus that it should not be held on general election day, which would have been impossible in the time left to the government anyway.

Nevertheless, the upshot of all this was that Labour included a commitment to an AV referendum in its manifesto, which meant that two parties went into the Coalition negotiations on the reform side of the argument, forcing the Conservatives to shift ground.

As it happens - and as Andrew Adonis confirmed this week - Labour negotiators were sympathetic to the idea of a referendum on three options: proportional representation (PR), the alternative vote (AV) and First Past the Post, albeit that they would not commit to supporting PR in a referendum, which the wider Labour Party would never accept. In the days leading up to the general election, I and others in the Number 10 team advised Gordon Brown to offer a two-part referendum, on the New Zealand model, with a yes/no question about whether to change the system at all, followed by a further ballot on which reform option to pursue, should the electorate have chosen reform. In the end, there was no final agreement on these issues in the negotiations, as it became clear that once the Conservatives had moved on the AV referendum question, any prospect of a Labour-Liberal Democrat deal was remote to the point of vanishing.

Why then has the Yes campaign apparently fallen so short? The important point about all these events over the last two years is that no significant popular mobilisation has taken place to underpin, nourish and channel energy into political and democratic reform. Nothing akin to Charter 88 or the Scottish Constitutional Convention in the late 1980s and early 1990s has animated recent reform efforts. There has been little in the way of intellectual meeting of minds in the broad penumbra of supporters on the reform side. The public has simply not been engaged, despite the best efforts of the Yes campaign.

In contrast, the No camp, which is essentially the right of British politics combined with Labour tribalists, has pulled all its forces together. Indeed, what I believe to be the intellectual weakness of the contemporary right in Britain is offset by organisational strength and campaigning effectiveness across a range of inter-related groups. It does not command ideological hegemony, as it did in the 1980s, but makes up for it in unashamedly hardball political mobilisation - almost as if an ideological insecurity, and the wound of failing to win an outright majority in 2010, has been sublimated into concerted political aggression. It's rough stuff, but progressive opponents need to learn how to respond to it, not complain about it.

More importantly, however, lessons need to be learnt about the politics of reform. In other parts of the world - notably Canadian states - citizens have been directly involved in randomly selected deliberative assemblies with the task of framing questions and potential referendum options on electoral reform. Popular legitimacy can be built into the process from the start, not sought at the end.

For progressive political forces, the referendum experience must also be taken as an opportunity to reflect on how to marshall deeper forces for change. The Liberal Democrats are now too weak to drive change on other issues, like party funding, without broader progressive support. They have to reach out - not just within the Coalition on things like Lords reform, but beyond it. For its part, Labour remains too divided on these issues, and needs to go through a process of renewal akin to that undertaken between 1987 and 1997. Civil society needs to be engaged, as it was in Scotland prior to devolution. And wider intellectual groupings need to be formed, to sit alongside campaigning organisations. Even if electoral reform for the Commons is now off the agenda for the foreseeable future, momentum for wider change must begin afresh.

This piece was originally published on Nick Pearce's ippr blog.