Prime Minister David Cameron speaking at the Centre for Social Justice, October 2012

Speaking at the Centre for Social Justice in October 2012 Prime Minister David Cameron announced a radical acceleration of privatisation across the justice system. He said: “With payment by results, your money goes into what works: prisoners going straight, crime coming down, our country getting safer. It’s such a good idea I want to put rocket boosters under it.”

Today’s report by the National Audit Office (NAO) reveals that the government is not even monitoring whether payment by results works, and explodes the myth that it is a magic bullet solution to intractable social problems.

Payment by results (PbR), also known as outcome-based payments, is a way to deliver services where all or part of the payment depends on the provider achieving specified outcomes.

The public spending watchdog comes up with three overarching findings after examining the recent history of PbR schemes including the Work Programme and the privatisation of the probation service, otherwise known as Transforming Rehabilitation.

Firstly, the NAO finds that payment by results is not suited to all public services, and, if applied inappropriately, there is a risk that either service quality or value for money can be undermined.

Secondly, it says commissioners should justify their selection of payment by results over alternative models. Piloting it can help in this process.

Thirdly, the report describes in some detail how payment by results is a technically challenging form of contracting – one which has attendant costs and risks that government has often underestimated.

So, as the Howard League for Penal Reform has been warning for years: not a magic bullet. Chances of shooting oneself in the foot: high.

The government has staked almost £15 billion worth of contracts on the payment by results magic bullet since 2008/9. The Ministry of Justice is particularly keen. The department’s Transforming Rehabilitation programme represents just over 21 per cent of the entire budget for PbR across government. That’s £3 billion over seven years.

Outsourcer G4S: evidence to Justice Committee, January 2012

The Ministry of Justice has had a somewhat Jekyll-and-Hyde approach to PbR over the years. Under Kenneth Clarke, the MoJ focused on carefully piloting PbR in two private prisons – Doncaster and Peterborough. Such an approach is commended by the NAO in their report. But then Chris Grayling took over the department and binned all caution to push a national roll-out of PbR before the pilots’ results were fully understood.

We wait to see what Chris Grayling’s successor, Michael Gove, will do. But it seems unlikely that he will throw Grayling’s reforms into reverse, not least because of penalty clauses built into the contracts would punish the taxpayer.

If Michael Gove were to consider expanding PbR into other areas of the justice system, he might seek advice from the Cabinet Office or the Treasury. But, as the National Audit Office report reveals, neither of these departments monitor how payment by results is operating across government. Without a systemic collection of lessons or any evaluation of whether it this system is delivering its claimed benefits, there is no central source of comparative evidence to inform the choices that policymakers might make.

Meanwhile, the rhetoric marches on without the evidence. We are told that PbR is the ideal method of tackling difficult social problems that lack ready solutions, such as “reducing reoffending”. Yet PbR providers admitted to the NAO that, given the risks associated with this method of commissioning, it is best suited to “issues to which there are known solutions and where the commissioner’s overarching aim is to reduce costs”.

PbR mechanisms create perverse incentives, such as the tendency to encourage providers to prioritise those service users who are “easier to help”. The watchdog describes the risk that providers simply focus on “quick wins” that have no lasting impact on longer-term outcomes. Providers may seek to maximise their profit margin on the initial fees alone by cutting costs as far as possible, then largely ignore whatever payment by results element there may be to be earned.

The NAO being the NAO, however, the report stops short of damning government policy. In particular, the NAO take certain claims made by the Ministry of Justice at face value. In doing so, further grounds for criticism are missed.

Transforming Rehabilitation is too recent to be examined in detail and so the NAO concentrates on the Peterborough and Doncaster pilots, both of which continue to be evaluated despite the ministerial bandwagon moving on. Because the model being rolled-out nationally is different from either pilot, you may think their effectiveness or otherwise is meaningless. But it is fair to say that these pilots were intensive versions of what is now being watered down for widespread consumption across probation.

Those schemes offered ‘through the gate’ support to people leaving prison, with an emphasis on short-sentenced prisoners who traditionally have not received such support.

Peterborough used a ‘social impact bond’ model where charities and social investors put money into the project. Two groups, or ‘cohorts’, of prisoners leaving the jail would be tracked. If there was a 10 per cent reduction in the frequency of reconviction events for the first cohort against a matched national control group, then a payment would be made to the investors. If a 7.5 per cent reduction occurred across the two cohorts (on average) then a second payment would be made.

Doncaster, on the other hand, saw the contractor Serco aiming to achieve a 5 per cent reduction in reconviction rates to cover their core contract costs. If the prison could reach a 6 per cent reduction or higher, then the company would earn additional payments.

Both pilots did show reductions in reconviction measures, but not enough to justify the payment by results approach. Peterborough managed an 8.4 per cent reduction in the first cohort and so did not trigger its payment. The NAO blandly report that the second cohort is 2.3 per cent below its comparator group — an interim assessment which would mean that Peterborough also fails to trigger the second round of payments.

Doncaster’s first cohort meanwhile did meet its basic target of 5 per cent, as Serco managed 5.7 per cent. But it did not trigger additional payments and the second cohort is also not performing as well in the interim assessment – with only a 3.9 per cent reduction managed thus far.

Both pilots were helped by the exclusion of police cautions in calculating reconvictions, despite cautions being included in regular proven reoffending statistics from the Ministry of Justice.

At best, then, the pilots show that doing something is better than nothing, without any evidence that PbR is the best way to do that something.

It is time for the government to scrap the rocket boosters and apply some scrutiny to payment by results.