The west must understand the Islamic State's worldview, and accept its own failings, if it is to meet the challenge.
In the early months of 2014, there was little particular concern in the west about the movement then known as the Islamic State in Iraq and the Levant (ISIS or ISIL), and now as the Islamic State (IS). That changed radically with the group's rapid advance through parts of Iraq, including the capture of the city of Mosul, in the first half of June. Now, even greater concern is raised by the gruesome killings - staged for propaganda purposes - of two journalist hostages, James Foley and Steven Sotloff.
There is some caution over an immediate military response to the murders, amid recognition that this may be what Islamic State wants. Indeed, the murders may be a direct incitement to just such action (see "Second execution video shows that Islamic State has a grim strategic plan", The Conversation, 3 September 2014).
It's clear that the debate about how western states should respond needs to be based on understanding the movement's current status and motivation. At least five elements are important here. First, IS is well organised and coherent in the running of the territory it now controls - an area about the size of the UK and with well over 4 million people under varying degrees of control.
Second, it works readily with other groups in a manner that might suit its purpose in the short term. These include Ba’athists and Sunni clans in Iraq that were strongly opposed to Nouri al-Maliki's government in Baghdad, which favoured Iraq's Shi’a majority. That may change now that al-Maliki has been replaced as prime minister by Haider al-Abadi, but perhaps not for many months.
Third, IS's paramilitary ability is enhanced by years of fighting in Syria and (even longer) in Iraq. In the latter case, moreover, its combat was against American troops who were both well-armed and highly motivated by the need to respond to what they saw as a terrorist insurgency directly connected to the 9/11 attackers. A feature of the United States occupation of Iraq was the detention of tens of thousands of Iraqis without trial, often for years on end. The squalid and congested prisons were hot-houses for radicalisation, and many young men were more than willing to join the movement after their release.
Fourth, IS is sophisticated in the use of propaganda to target potential supporters and recruits. Its presence on the new social media, and its holding of hostages as a counter in relation to western military action, are examples.
Fifth, and perhaps most important, is IS's underlying idea - the creation of a new caliphate - which has proved capable of striking a chord in the minds of disaffected Muslims, mainly young and male, and including those with little or no direct experience of the region.
This ambition draws on the historic desire to establish a unitary religious-political entity of a type known in the Islamic world over the past 1,400 years. The last caliphate was abolished as part of the fall of the Ottoman empire in the early 1920s, though the classic reference-point is more often the Abbasid caliphate which was centred on Baghdad from 750 CE and peaked around the mid-10th century.
That caliphate was powerful: it stretched across much of the modern day Middle East, was the world's most important centre of civilisation (with impressive achievements in astronomy, art, architecture, medicine and many other areas), and was relatively benign in political terms (with prominent Jewish and Christian communities in the larger cities).
Such an exemplar is quite unlike anything the extreme and puritanical IS envisages. But the movement's success in promoting the idea of a new caliphate does exploit a deeply inbuilt perception that the Muslim world has been in retreat for hundreds of years in the face of western expansionism, and that this must be reversed. It seems a remarkable worldview when the typically western fear is that of a rampant Islamist entity challenging the west at every turn. There seems little possibility of any meeting of minds.
A time for reckoning
This makes it even more advisable to delve a little further into how this narrative spreads, especially among some disenchanted and marginalised young men. There are two aspects to the process. The first is the repeated claim that the United States and its allies are rigidly determined to take over the entire Islamic world. This “far enemy”, the narrative goes, has been barbaric in its suppression of independent states; it has also specifically aided one state, Israel, in pursuing this aim even to the heart of the Islamic world (not least by controlling Islam's third holiest site, Haram al-Sharif). What is being argued here is that a crusader-Zionist plot threatens the integrity of Islam; the plot must be fought and defeated, however long this takes.
The second is that much of the action since 2001 can easily be represented as just this kind of assault. It includes regime termination in Afghanistan, Iraq and Libya as well as many instances of barbaric behaviour such as the marines' reprisal air-raid on Fallujah early in 2004 (see "Between Fallujah and Palestine", 22 April 2004). Furthermore, the crusader-Zionist element has received a substantial boost by the death and destruction in Gaza in July-August 2014, which were made possible by billions of dollars of US military assistance to Israel - the fruit of a cosy, decades-long relationship between the two countries (see "After Saddam, no respite", 19 December 2003)
The movement thus contains an eschatological dimension but also encompasses the traditional acquisition of power, thoroughly imbued with male control. Even if much of its worldview is gross exaggeration, it has enough truth to provide it with a dangerous air of authenticity and attract widespread support.
On this basis, the Islamic State seeks to establish an unyielding caliphate in response to a perceived western threat. If the west is ever to get to grips with the challenge, it will have to confront some of its own grievous mistakes and even worse behaviour. That will be very difficult, but it is essential if there is to be any way forward.