The Brazilian authorities have every reason to be grateful that the soccer world cup in Germany arrived just in time to divert people's attention from the crisis of violence and confrontation in the country's (and South America's) largest city, São Paulo. The two laboured victories against Croatia and Australia have left Brazil's people and media worrying about their team's performance, hopeful that the players will raise their game when they need to, and as absorbed as ever in the permutations and personalities of the squad (is Ronaldo overweight; is the 36-year-old Cafu past his best; can the team cope with the Argentinean threat?).

For a country engrossed in the month-long spectacle, these are the questions that matter. But when it is all over, and Brazil's players return home in triumph or disgrace, Brazilians as a whole will return to face a far starker and less glittering challenge: the condition of the country's criminal justice and prison systems, and the deeper realities of social exclusion and inequality this system is founded on.

Sue Branford reports from Latin America for the BBC and the Guardian. She is co-author (with Jan Rocha), of Cutting the Wire: the Story of the Brazilian Landless Workers' Movement (Latin America Bureau, 2002) and (with Hugh O'Shaughnessy) of Chemical Warfare in Colombia: The Costs of Fumigation (Latin America Bureau, 2005)

Also by Sue Branford on openDemocracy:

An urban civil war

When a nation is absorbed in the fate of its yellow-jerseyed heroes a continent away, it's not the moment for the Primeiro Comando da Capital (First Capital Command / PCC), São Paulo's most powerful criminal gang, to put on another show of strength.

Indeed, the jailed PCC leaders will be cheering on the Brazilian side with their compatriots. Along with mobile-phones and DVD players, they have widescreen, plasma television sets in their comfortable cells; and like millions of citizens on the outside, they will toast Brazilian victories (or drown sorrows) with a good few glasses of Brahma lager.

Yet the reprieve for the authorities may be shortlived, and a repeat of the wave of urban attacks and prison riots from 11-19 May 2006 not long in coming. Then, the PCC stunned Brazil when it organised a wave of violent actions across the state of São Paulo, Brazil's industrial heartland. The authorities had been alerted by informers that the PCC was planning a prison rebellion for 13 May – Mothers' Day in Brazil, when family visiting regulations are relaxed – and thus ordered the transfer of more than 500 PCC members to a high-security prison. But the PCC also has its informers inside the security services; it learned of the plan, and at short notice coolly brought forward its planned revolt by three days.

On the night of Thursday 11 May, riots erupted in almost 100 prisons. At the same time gang members, armed with machine guns, attacked police stations all over São Paulo state. At the weekend, they continued the operation: bombing police stations, destroying city buses, and ambushing police vehicles.

By the morning of Monday 15 May, the 19 million inhabitants of São Paulo were understandably jittery. When rumours proliferated on the internet that the PCC was about to unleash an even bigger and more violent round of attacks, people panicked. Schools, offices and shops shut at midday and people headed for home amid enormous traffic jams.

In fact, the PCC had by then called off its attack, apparently after making a secret deal with the authorities. Even so, what for many Brazilians turned out to be the worst aspect of the whole affair was still to come. The military police, infuriated by the death of some forty policemen and prison guards in the PCC campaign, launched its own retaliatory killing spree. Its intention from the beginning was simple and brutal: to kill PCC members or sympathisers.

A message on one police website read: "For every Mike ["military policeman" in the organisation's argot] who has fallen, two criminals must die." Another was even more sinister: "It's time to cleanse São Paulo, to get rid of this filth in our midst." Over the next few days 122 "suspected criminals" were killed in the city. Eye-witness accounts reported that men dressed in black and wearing hoods drove into shanty-towns and shot people sitting in bars or outside their houses. Antônio Funari, the São Paulo police ombudsman, said that at least twenty-eight of these victims had no criminal record.

The episode had a huge impact on the city's population. Before 11 May, many paulistas – particularly among the middle classes – had scarcely heard of the PCC. They now know that it is a large organisation with 140,000 members in the state's jails and another 500,000 in the wider society; that it is partly funded by a well-organised system of monthly contributions with different rates for prisoners and free members; and that it has built up a large network of informers, from lawyers to taxi-drivers.

The scale of the gang's activities and ambitions is remarkable. A federal police report says that the PCC is responsible for at least 20% of the transport and high-value crimes in the country. In just one spectacular robbery, in Fortaleza (northeast Brazil) in August 2005, the gang seized three tonnes of bank notes with a total value of 164.8 million reais (£40 million).

The PCC's power: three sources

Why has the PCC been so successful? This is the most painful question of all for many middle-class paulistas. Amid their soul-searching, three answers suggest themselves.

First, the PCC's main recruiting ground is the prisons. It has been known for years that Brazil's prisons are a humanitarian disgrace: overcrowded, with cells meant for three people containing twelve; rife with drugs; lacking any rehabilitation policy; and characterised by administrative chaos that leads to many prisoners being detained long after they have completed their sentence.

Heidi Cerneka from the Catholic church's prison ministry – who in 2002 commented that "(public) space" in São Paulo "is identified more and more with violence, danger and abandonment" – points out that the PCC is the only organisation that has done something for these abandoned people: "Through their campaigns they have reduced the level of violence in prisons; they have won better visiting rights, even paying for coaches for family members who have to travel long distances, and they have provided defence lawyers for their members."

The PCC is also utterly ruthless, Cerneka says: "They defend to the death their members but they have no qualms in assassinating anyone who betrays them or owes them money."

Second, the authorities are making this bad situation even worse. Paulo Sérgio Pinheiro, one of Brazil's most committed human-rights defenders, has been saying for years that the government should dismantle the repressive public-security bodies set up by the military dictatorship in the late 1960s. The military police were created in that era to combat leftwing guerrillas, and today use the same tactics to target criminals in shanty-towns; it is a refocusing of their efforts without a rethinking, and one that fails to take account of the very different nature of the security challenge.

The military police's attitude can be gauged from the slogans that its officials have adopted. Saulo de Castro de Abreu, the secretary of public security in São Paulo state, has repeatedly said: "A good bandit is a dead bandit buried in the moon." Ruy Ferraz Fontes, a top detective from São Paulo's department of investigation into organised crime, has a sign printed on his door: Direitos humanos para os humanos direitos ("Human rights for right-thinking humans"; if the Portuguese pun does not translate well, the sentiment is plain). The words reflect and fuel actions: the military police are accused year after year of the systematic flouting of human rights, including the cold-blooded murder of hundreds of suspected criminals.

Third, an even more serious problem lies behind the expansion of the PCC. Brazil is one of the most socially unjust societies in the world. Around 46% of national income goes to the richest 10% of the population, while the poorest 50% earn only 13.3%. Moreover, Brazil's social (and racial) apartheid is paralleled in the existence of two systems of justice: one for the well-off and another for the poor. Members of the (largely white) middle and upper classes are very rarely sent to a common jail. As Brazilians say, prisons are for pobres, pretos e putas (the poor, black people and prostitutes).

The lot of the poor is a routine denial of justice. In early June 2006, cross-fire between the police and a criminal gang in a school in a Rio de Janeiro shanty-town wounded seventeen children. When the mothers arrived, the school was a scene from hell: wounded children screaming, panic-stricken youngsters hiding under desks, blood on the walls, desks and clothes. Yet there was hardly any investigation or follow-up, and the story was scarcely reported in the papers. No one expects those responsible to be punished.

I myself had a maid from a São Paulo favela whose husband and son had tried to protect the daughter of the family's neighbour from sexual abuse by her father. The abusive neighbour killed the two men. The devastated maid repeatedly complained to the police, but the neighbour was never even arrested.

Two worlds

President Luiz Inácio Lula da Silva, the centre-left president inaugurated in January 2003, has helped the poorest of the poor. A greatly enhanced social-welfare system has meant that the income of the poorest 10% of the population has risen by 23% in these three years, far more than the national average. This policy will almost certainly ensure that Lula is re-elected for a second term in October 2006. But the president's extremely cautious economic policies – under the guidance of his ex-finance minister Antonio Palocci (who resigned on 28 March 2006 after being implicated in the convulsive mensalão [vote-buying] scandal which dominated the country in 2005) have produced low growth levels and the creation of few new jobs.

The socially excluded in Brazil remain excluded, and crime is growing. Around 340,000 Brazilians are behind bars today; the number is expected to rise to 500,000 by 2008. This scale of increase suggests that Brazil should be opening a new prison every fortnight. There is no sign of this, guaranteeing a further deterioration of jail conditions and a recurrence of rioting.

The Primeiro Comando da Capital offers the predominantly young men caught up in its activities a kind of future, something that official Brazilian society fails to do. The PCC's income, derived largely from robberies and drug-trafficking, is estimated at approximately one million reais (£244,000) a month. It is reported to be opening petrol filling-stations and transport companies to facilitate the laundering of these funds.

Millions of poor, black Brazilians will passionately cheer their country's football team if it caps its efforts in Germany by retaining the world cup. They can see in the exploits of talented youngsters from modest backgrounds such as Ronaldo, Ronaldinho and Kaka a projection of their own dreams. These skilful and fortunate few have found another route to riches, but it is a melancholy reality amid soccer's joyful carnival that many Brazilians left behind have few options of survival, protection or escape except to join gang networks like the PCC.