Delta employees protest human trafficking. Delta News Hub/Flickr. CC (by)

In November 2014, the Walk Free Foundation launched the second edition of their flagship Global Slavery Index. The press release for the launch declared the index to be ‘the most accurate and comprehensive measure of the extent and risk of modern slavery’, building upon an ‘improved methodology’ including random-sample surveys from 19 countries. This emphasis on methods was significant, because the inaugural 2013 index had been repeatedly criticised for using unreliable, incomplete and inappropriate data. Despite this focus on improved methods, the second version of the index once again attracted sustained criticism. One of the sharpest challenges came from Anne Gallagher, who reported that the index contained ‘critical errors of fact and logic’.

While Walk Free wouldn’t necessarily agree with these exact sentiments, they have nonetheless acknowledged that the index suffers from a number of limitations.  These problems are not unique to the slavery index, but also apply to global estimates of slavery and trafficking more broadly. It is not our intention to revisit these specific methodological issues here. Instead, we are interested in why non-governmental organisations (NGOs) such as Walk Free have continued to generate statistical information that they know to be highly suspect, and why numerous third parties have continued to reproduce these types of statistical claims despite also knowing the same.

The Global Slavery Index, global benchmarking, and the politics of numbers

To help answer this question, we first need to consider the larger context. The Global Slavery Index is not a new innovation, but can instead be best understood as the extension of an already well-established template to a new topic. Over the last two decades, there has been remarkable explosion in the prevalence of national, regional and global benchmarks, such as indices and rankings. Some notable examples from a much larger trend include the Fragile/Failed States Index (Fund for Peace, from 2005), the Corruption Perceptions Index (Transparency International, 1995), the Climate Change Performance Index, (Germanwatch & Climate Action Network Europe, 2006), and the more venerable Freedom in the World report (Freedom House, 1972). In this specific piece we are concerned with NGOs, but it is important to note that this trend also involves governments (e.g. Trafficking in Persons Report, 2001), international organisations (Human Development Index, 1990), and corporations (PRS Group, 1979).

The Global Slavery Index closely adheres to established strategies and conventions. Complex social, economic and political phenomena—such as state stability and discrimination—are made both easily accessible and globally commensurable through radical simplification and ‘guesstimation’. Distinctive qualities are converted into numerical quantities, which are then compared and assessed in terms of orders of magnitude. Challenging and contested concepts—such as slavery—acquire fixed and unproblematic meanings, which are presumed to be universally applicable irrespective of cultural context. Having crudely converted the social world into numerical values, the Global Slavery Index then goes on to assign countries a ranking, with a ranking of one being the worst and a ranking of 167 being the best. These rankings are further organised into numerous tables, both global and regional, and are also found alongside ‘heat maps’, with shades of green, yellow and red being assigned to countries on a map.

Heat maps and ranking tables from the Walk Free Foundation's Global Slavery Index 2014. Fair Use.

In addition to seeking to measure prevalence, the 2014 index assigns governments a grade based upon their performance in combating slavery (i.e. AAA, AA, A, BBB etc.), and uses a composite scale in an attempt to gauge vulnerability (ranging from one to 100). Here, as elsewhere, the Global Slavery Index presents its findings in a similar fashion to other ‘benchmarkers’, who also routinely make use of rankings, heat maps, grades and scales.  In keeping with conventions, Walk Free makes further reference to a ‘top ten’ list, which is a popular device used to draw attention to the ‘best’ or ‘worst’ performers. In a world where school league tables and other metrics have become pervasive, the idea of ranking countries based on performance is a concept with which we are already familiar.

Numbers are fundamental to the political appeal of global benchmarking. Unlike words, which require interpretation, numerical claims are widely held to encapsulate unbiased facts. We have a tendency to fixate on specific numerical claims, which create ‘anchoring effects’ by establishing high profile ‘information shortcuts’ that shape how we then approach specific topics. Take, for example, the still popular—yet also highly speculative—claim that there are 27 million slaves in the world today. This was first published in the late 1990s, and then subsequently acquired the status of a timeless ‘fact’ via public repetition. Also important here is the capacity of numbers to present information in a format that is easily accessible to non-expert audiences, who might otherwise be overwhelmed by contextual details. The complex backstory behind how specific numbers are produced routinely gets lost once they are put to into the public domain.

The global slavery index, brand recognition, and the marketplace of activism

The political properties of numbers have contributed to demand for more and more numbers. This demand comes from journalists, politicians, administrators, activists and many others. Whenever there is significant demand, then supply is likely to follow. It is within this larger context that benchmarking has emerged as a popular strategy amongst NGOs. Thanks to the remarkable proliferation of NGOs since the 1970s, activists have been increasingly forced to compete with their peers for audiences, allies and investment. Benchmarking can serve a number of purposes within this competitive marketplace. Most notably, benchmarks can help to make a political case that the cause being benchmarked is worthy of concern, raise the profile of the organisation involved, and also promote their own vision of what ‘the cause’ looks like. Recognising that ‘first movers’ such as Freedom House have been able to raise their profile and reputation, NGOs such as Walk Free has turned to benchmarking to help build their own brand, and to promote their own conception of how slavery and trafficking should be understood.

As anyone who works on slavery and trafficking can tell you, the number of NGOs focusing on this topic has expanded tremendously within a short space of time. Within this crowded marketplace, the Global Slavery Index advances the Walk Free brand on a number of fronts. First, and most obviously, the index helps to build its public profile. Benchmarks are catnip for journalists, so publishing a benchmark typically attracts a huge amount of media attention both for the cause and for the organisation producing the benchmark. Moreover, benchmarks also offer a useful platform from which to seek the endorsement and support of the rich and powerful (the Global Slavery Index has been endorsed by Hillary Clinton, Tony Blair, Gordon Brown, Julia Gillard, Bill Gates, Richard Branson and Mo Ibrahim, amongst others).

Second, benchmarks have proved to be effective as a tool for establishing a reputation for authoritative expertise and credibility, which may or may not be warranted. In the case of the Global Slavery Index, Walk Free has sought to bolster its credentials by bringing in Kevin Bales, who is best known for the figure of 27 million slaves mentioned above. Finally, benchmarks have also proved to be popular amongst funders, and NGOs have long been adept at working out what funders want and tailoring at least some of their activities accordingly (this is probably less of a concern for Walk Free, which has private funding). Thanks in part to the digital revolution of the last two decades, benchmarks have also become relatively cheap and easy to produce and disseminate. It is not necessary to have years of expertise in the field, or knowledge of local languages. What is instead primarily required is the capacity to compile different forms of secondary data, which often in turn involves aggregating information from one benchmark in order to create another.

The cost of radical simplification

Walk Free is aware that the Global Slavery Index rests upon a very limited foundation, but it has calculated that the advantages of having a slavery benchmark outweigh any likely costs and complications. Most third parties who use the index have made the same call. As we have demonstrated, the main advantages associated with having a slavery index are political and organisational, rather than analytical. While benchmarks tend to be good at generating brand recognition and publicity, this success relies upon the popular appeal of ‘information shortcuts’ that sidestep all kinds of substantive issues.

It is not possible to have a Global Slavery Index without reducing different forms of complex and case-specific phenomenon into more easily comparable and accessible numerical values and statistical tables. Huge amounts of depth and detail invariably gets ‘lost in translation’ as a consequence of this process of radical simplification (while unreliable and incomplete data can in turn acquire the status of ‘fact’, which they really do not deserve). It is this type of detail that is essential to formulating and applying effective policy solutions. In addition, benchmarking also tends to have the effect of assigning responsibility for ‘success’ and ‘failure’ within countries, and thereby end up failing to engage with forms of interdependence between different parts of the globe. As Siobhán Mcgrath and Fabiola Mieres have argued, the ranking system used by the Global Slavery index ultimately ‘implies that the blame should be placed squarely on the national governments of the less ‘developed’ countries for the plight of their citizens’, and in turn casts more ‘developed’ and ‘civilised’ countries in the role of ‘rescuers’.  

While Walk Free and other NGOs are aware of at least some of these problems, they are reluctant to say too much about their overall ramifications. This could end up casting doubt upon their statistical findings (perhaps slavery isn’t as big as reported), their vision of ‘the cause’ (perhaps slavery isn’t the best frame of reference) and/or their organisation (perhaps Walk Free doesn’t have the expertise it claims). This contributes to an environment where there is widespread reluctance to seriously question statistical ‘facts’ that rest upon shaky foundations, and which in turn offer an ineffective—or counterproductive—foundation for thinking about more effective policy interventions.  

The Global Slavery Index is a great publicity tool, but it is not very good at offering the type of nuanced analysis required to guide either understanding or policy. A trade-off between the political and analytical has taken place here, and this needs to be publicly recognised. Radical simplification often comes at the cost of contextual understanding.

This article draws upon material from a research project on global benchmarking. Some of the outputs of this project will soon be published as a special issue of Review of International Studies on ‘The Politics of Numbers: The Normative Agendas of Global Benchmarking’.