The Notting Hill Carnival is the largest street festival in Europe, attracting two million people. Taking place annually over the August bank holiday weekend, the Carnival transforms a rectangle of West London bounded by Ladbroke Grove to the east and the Great Western Road to the west, and stretching from Notting Hill Gate in the south to the Harrow Road in the north.

The Carnival parade is great fun – a succession of floats, lorry mounted sound systems, and platoons of peacock garbed masquerade ‘camps’. These are the elements familiar from the founding carnival traditions of the Atlantic Rim: New Orleans, Mardi Gras, and the Carnivals of Port of Spain, Trinidad, and Rio, Brazil. The crowd joins in the parade as it weaves round the circuit, and the costumes, though not as revealing as in Brazil, are often spectacular. If it lacks the participatory abandon of a Trinidadian carnival, it nevertheless retains the rich flavour of a community event, as compared to the extravagant floats sponsored by ATT and Coca Cola, which have ruined New Orleans’ annual shindig.

In its promotional imagery, as in popular imagination, the Notting Hill Carnival is confirmation of the health and vitality of post-colonial, multicultural Britain. Images of young people, black and white, in shared celebration, of policemen grinding with passing ragga girls, suggest a trans-racial coalition. Such an interculture surely exists in London, where the morbidities of parochial English racism have been rubbed away through day-to-day, banal contact across what W. E. B. Du Bois called the ‘color line’.

Yet all in London’s multicultural garden is not rosy. Carnival is no celebration of a pre-existing harmony, but an attempt to found a multicultural community, sometimes in the face of extreme adversity. The origins of carnival itself are contested. Despite the debt to Trinidadian carnival and music in the parade, carnival has always been more complex than a transplanted Trinidadian tradition.

Carnival is a blast, and I don’t want to put anyone off going, but conflict has stained its history. Conflict between different organisers with different aspirations for its future, between divergent versions of British Caribbean identity, and most of all between those – black youth and the police – who make carnival what it is.

The 1950s

The dismal film, Notting Hill, effectively airbrushed its black community out of history. So you may not know that in the late 1950s, Notting Hill was a depressed and faded shadow of its former Regency splendour. The splendid terraces and grand avenues through which the carnival parade yearly winds, were, back then, suffering from extreme dilapidation. Most were subdivided into flats and let, at a premium, to Caribbean migrants. Slumlord, Peter Rachman, was only the most infamous Notting Hill property speculator exploiting the fact that black Caribbeans found themselves unwelcome in many other parts of this frigid, austere, still-recovering-from-the-blitz city.

The majority of Caribbean immigrants to the UK came from the two largest islands in Britain’s then crumbling empire – Barbados and Jamaica. Bajans, Trinidadians, Tobagans and those from Nevis and St Kitts, Antigua, the so-called ‘small islanders’, tended to settle close to each other in Notting Hill, whereas settlement in South and North London – Brixton, Tottenham- was primarily Jamaican.

As Professor Stuart Hall points out, it wasn’t until they came to Britain, and were greeted by a routine racism they had never expected, that these islanders with very different Caribbean cultures and histories, ‘discovered themselves’ as West Indians. If Britain was the ‘mother country’ as imperial propaganda had it, her colonial children – coming to the UK during post-war reconstruction and answering the call for labour as British citizens – soon realised she was an unfit mother, on whose love they could never rely.

In 1958, the first of many post-war ‘race riots’ occurred in Notting Hill. A sustained and unprovoked attack on black residents inspired by bully boys exploiting local tensions, it is significant in Black British lore as the point at which black Londoners – African, American and Caribbean – recognised the need to organise, fight back, and celebrate the commonality that had been forced upon them. ‘Black’ became a political identity.

The first carnival – a Caribbean ‘fayre’ staged in St Pancras town hall in 1959 – was born in this context. It was an attempt to galvanise London’s disparate black community. Arranged by the Trinidadian communist Claudia Jones, founder of the West Indian Gazette, the fayre embodied Jones’ recognition of the political force of culture. With something of the Harlem Renaissance emphasis on folk tradition (Jones grew up in Harlem), the early fayre drew on Trinidadian traditions of costume and the scurrilous political commentary of Calypso.

The 1960s

In the early 1960s, Jones’ festival, before her untimely death in 1964, managed to establish itself as an annual event.

Meanwhile, another annual event was emerging in Notting Hill itself. Rhuane Laslett, a social worker from the East End who worked in the area, had for some time been determined to represent the cultural diversity of the area – Ukrainians, Spanish, Portuguese, West Indians – in a street festival. Following Jones’ death, her indoor carnival was fused with this and, in 1965, the first Notting Hill carnival, a rag tag parade with no defined route but plenty of enthusiasm, weaved through the streets of Notting Hill. Within the cultural mix that Laslett envisioned, Trinidadian traditions played a prominent role – flamboyant costumes, Calypso, Caribbean food and soon the emblematic Steel Band.

But changes were afoot in London’s black culture that were to have a profound impact on carnival in the coming decade. Jamaican popular music was starting to emerge as a major global force. Since 1964, when Millie Small’s unlikely hit single ‘My Boy Lollipop’ stormed the US and UK charts, the Jamaican music industry had gone into overdrive, capitalising on the popularity of the new, razor-sharp ghetto style, ska. Personified by the impossibly cool former boxer Prince Buster, ska gave a modernist twist to Jamaican folk rhythm (mento), injected a new confidence into the Jamaican record industry, and found a new audience in the UK not only among Caribbean migrants but in the emergent white youth subcultures of Mod and Skinhead.

From 1967 on, when Desmond Dekker’s ‘The Israelites’ topped the UK chart, Jamaican popular music began to supplant not only the American jazz and soul that had been the staple diet of Black Londoners, but the folksy good-time rhythms of Soca and Calypso. Its political wit notwithstanding, Calypso soon ceased to cater to a post-1958 generation of black London youth for whom assimilation to a bankrupt racist system was no more an aspiration than it was an option.

By 1968, ska was evolving into ‘rocksteady’, a slower and more militant form of Kingston ghetto music. And by the early years of the 1970s, reggae emerged. The mod image of the ‘rude boy’ gave way to the dreadlocked Rastafarian, whose ‘conscious’ stance drew on explicitly Jamaican traditions of resistance to imperial power dating back over a century, combined with a millenarian religious credo. The music drew on West African nyabingi rhythms, mento and soul, to create a heady, uniquely paced bedrock for the sermonising of Jamaica’s new generation of socio-political orators. The songs of Bob Marley, Burning Spear, and Horace Andy, and the heavy productions of Lee Perry and Augustus Pablo, chimed precisely with the experience of Black Londoners – ‘stolen from the homeland’ (Africa), trapped in a ‘Babylon’, ablaze with racial conflict, misunderstood, reviled but unbowed.

The 1970s: sound systems

From ska onwards, the prime source for disseminating Jamaican music was not the live orchestra or steel band, but the sound system. Massive mobile hi-fis, the sound systems of Kingston legends, Duke Reid, Clement ‘Coxsone’ Dodd and Prince Buster, played records wherever they could set up – church halls, nightclubs or in the ‘yard’. Since the 1950s, Black London had developed a sound system tradition of its own, with figures such as Count Suckle and Duke Vin staging ‘dances’ across the city. But in the early 1970s, sound systems began to infiltrate the Notting Hill Carnival and, by 1975, when Jamaican Leslie Palmer took over, they had supplanted the parade and the Calypso bands as the biggest draw, the musical heartbeat of carnival.

The Carnivals of 1975 and 1976, in particular, were marked by confrontation between black youth and the police. Tensions had always been there. With some justification, black Londoners regarded the British police as forces of arbitrary oppression. Throughout the 1970s, the use of heavy-handed tactics such as the suspicious persons (sus) laws meant that there was scarcely a single black person in London who had not been stopped and searched by police, many numerous times. This, combined with frankly idiotic police containment tactics, made confrontation inevitable.

As journalist Gary Younge points out: ‘Massive in size, working-class in composition, spontaneous in form, subversive in expression and political in nature – the ingredients for carnival are explosive.’ (The Guardian Weekend, 17 August 2002). Events in 1976 escalated into a full-scale riot, during which the poorly-prepared police were rained with bottles and scattered by an angry crowd.

Carnivals in the later 1970s and 1980s continued to be marked by these tensions. Every year the police would come up with a new wheeze to channel, control and pacify the crowd – most of which had the opposite effect. Some years, barriers appeared on major roads, cutting the crowd in half; attempts were made to close the carnival before darkness fell, which led to frequent conflicts when they came to shut down specific sound systems; and the tactics of closing off exits in order to channel the crowd in certain directions always led to frustration and anxiety. This was by no means confined to black carnival-goers. Anyone present could feel the tension amongst the police and the inevitability of explosion.

But the history of carnival also marks out Britain’s multicultural evolution – an unplanned process Stuart Hall has called ‘multicultural drift’. Within the area bounded by the parade, forty-one official sound systems, as well as hundreds of domestic hi-fis dragged for the day out on to the stoop, blast out the splendours of what Paul Gilroy has called ‘Black Atlantic’ popular music: jazz, roots reggae, soul, R&B, house, Garage, hip hop, dancehall, techno, rhythms and blues. The genuine multiculture that has emerged around the music of afro-America, the Afro-Caribbean and in the past decades Afro-England is nowhere plainer in evidence.

Don’t get me wrong: I am always thrilled to come upon the carnival parade on my carnival wander. But it’s the sound systems that are the heart of the matter. That’s where revellers let go and get wild; where Brits of all complexions cast off the social inhibitions which shackle them at most other times; where the fundamental hybridity, the ultimate heady concoction of London’s multiculture, is finally, clearly understood.

Notting Hill Carnival draws on the music of the Black Atlantic and the forms of the Afro-Caribbean in particular. But it is forged in the local circumstances of London. It is something of which we, Londoners, can be justifiably proud.

We would do well to remember, too, that it has been born in struggle and conflict, and that the multiculture it represents needs to be renewed in every generation. It needs defending, not only from those who still view difference with suspicion and mixture as betrayal, but also from the corporate raiders who are keen to harness the vitality of multiculture for their own ends. Corona beer sponsored a float in this year’s parade; the gathering forces of commercialisation cannot be far behind.

‘Carnival’, like metropolitan multiculture itself, is not a product or a commodity. It is an achievement of real people, and of the most binding of arts, freely given: loud music on a sound system.