‘Desire of Codes’ installation, Seiko Mikami. Flickr/Ars Electronica (Ryuichi Maruo, YCAM). Some rights reserved.Next week, the UK government will publish its new Investigatory Powers Bill, the successor to the Snooper’s Charter (officially the Communications Data Bill) killed off by Liberal Democrats in the last parliament. This legislation is expected to be very wide-ranging, covering powers for the police and security services to listen in to phone calls, read emails, interfere with our computers, and for them and a huge range of other organisations to find out who we ring, text or email, where we are when we do it, and every website we ever go to.

It’s an important piece of legislation, with potentially huge impacts on our lives, in particular on the levels of privacy and security we can expect to have. However, since it will inevitably be somewhat technical, there is a real risk the debate will focus on grossly oversimplified claims and the invoking of terrorists, paedophiles and the like. From personal experience, the quality of real analysis of these issues in parliament is not anything like as high as is needed, with a great tendency for parliamentarians simply to give way to the demands made, however thin the evidence for them.

Fortunately, when we demanded a sunset clause in DRIPA, the emergency legislation that dealt somewhat with these issues in 2014, we also secured a commitment that the replacement legislation would be presented as a draft, to be scrutinised in detail by a special committee of MPs and peers over a number of months, before being published and debated as normal. This gives a real chance to improve the legislation.

It’s somewhat brave of the home secretary to agree to this scrutiny. When the equivalent committee (on which I served) analysed the last version, we concluded unanimously that the bill went “much further than it need or should”, paid “insufficient attention to the duty to respect the right to privacy” and, perhaps most damningly of all, described some of the information provided by the Home Office as “fanciful and misleading”.

There is a real risk the debate will focus on grossly oversimplified claims.

So as we await the legislation, what do we expect to be in it – and what should be there? And even more fundamentally, does there need to be legislation at all?

The last of these questions is the easiest. We currently have a piece of legislation called RIPA (the Regulation of Investigatory Powers Act), which is generally agreed to be outdated, and written in such a way as to deliberately confuse people. It’s the legislation that allowed powers originally billed as essential to fighting terrorism to be used by local councils to find out if children were going to their catchment-area school. It’s broken and needs replacing.

And worse, many organisations have extra powers beyond those in RIPA. In 2001, the Department for Work and Pensions was given its own powers to access communications data, outside the control of RIPA, and clause 94 of the Telecommunications Act 1984 gives completely uncontrolled powers to do almost anything in the interests of national security or to help another country. 

And as we know from the Snowden revelations, there are many, many loopholes that are regularly exploited in the current legislation. Of course, we need security and intelligence services, and it should be no surprise that spies spy – but as the former Director of GCHQ has made clear: "Democratic legitimacy demands that where new methods of intelligence gathering and use are to be introduced, they should be on a firm legal basis and rest on parliamentary and public understanding of what is involved”. We should set the rules publicly, and check they are being followed, although we don’t need to know exactly what the security services are doing on every day.

So yes, we do need legislation, and it has to be done soon. DRIPA expires at the end of 2016 (or sooner, depending on a pending court case), so there’s not much time left to avoid another rushed job. And it should be all-encompassing, so there is one document that sets out what is and is not allowed, with no loopholes or hidden gaps. Everything else giving powers in this area should be expressly repealed.

The first element it must contain is rules for dealing with the content of communications – listening to phone calls, reading emails. It’s an important thing for the security services to be able to do, but also very intrusive. It must only be used on people who are suspected of serious wrongdoing. Currently, it is up to ministers to sign off warrants to do this, giving them a huge level of power. As recommended in the important review by David Anderson, the Independent Reviewer of Terrorism Legislation, this should in future be done by a judge. Judicial authorization is an important democratic safeguard, and is done by every other member of the ‘Five Eyes’ who share intelligence information.

Flickr/Ben Raynal. Some rights reserved.There is also a technical challenge to reading online content. It’s increasingly likely to be encrypted. David Cameron has insisted that there should be no “means of communication between people which we cannot read”. As I’ve written previously, this is simply impossible to achieve without also making it possible for criminals, foreign powers and others to also read it. No backdoor stays perfectly secure.

It now seems that the government has realised the flaws in the PM’s suggestion, so will not seek to ban encryption, though they may well stick to existing powers requiring companies to decrypt material they hand over. It’s not clear what happens if that is simply impossible, and it has never been enforced.

One alternative way of getting information without intercepting communications is to hack into devices themselves (so there’s little point encrypting your text messages if they are stored in a readable form on your phone!). This technique, known as Computer Network Exploitation (CNE) has hitherto been very secretive, and it has only recently been admitted that this is done by the UK security services. It is essential that the powers to do this are written out in this legislation, where anyone can see what the rules are. If it is to be allowed, it needs to be under tightly controlled circumstances where an individual is strongly suspected of criminal involvement.

The law must expressly rule out generic CNE on individuals not suspected of any wrongdoing. In addition, the safeguards and controls should be at least as strong as for interception. After all, I can avoid something private being leaked by just not sending an email, but if the state claims the right to hack my webcam, to turn it on when I don’t want it on, that can be far more intrusive. And this happens. GCHQ collected webcam images from millions of Yahoo users. This probably makes them the world’s largest possessor of amateur pornography.

Judicial authorization is an important democratic safeguard.

Most people are intrinsically uncomfortable about someone reading their personal messages, but even if the content is not read, just knowing who sent a message to whom, when it was done and where they were can be of great value to the police and others, and can also be very intrusive into people’s private lives. As things currently stand, a record is kept for 12 months of every time you send a text message, including where you were and who you sent it to.

There’s no doubt that this information can be useful to the police and security services, but also no doubt that it can be very intrusive. There are hundreds of thousands of requests each year for this information, and it can build up a very detailed profile of our behaviour. Currently, judicial authorisation is only required when a local council requests data (after much criticism of the way councils were using it), and there seems no good reason not to require judicial authorisation more widely for more intrusive requests.

There’s also little evidence that the data needs to be kept for anything like as long – effectively, we’re treated as suspects for 12 months, with information stored just in case. Is that really needed? While it can be useful to know, for example, who someone spoke to just before they died, that information can be obtained in days, not months.

‘Desire of Codes’ installation, Seiko Mikami. Flickr/Ars Electronica (Ryuichi Maruo, YCAM). Some rights reserved.In the failed 2012 Snooper’s Charter bid, it became clear that the Home Office wanted even more information to be collected than was previously the case. In particular, they wanted complete web-logs to be kept for 12 months, listing every website you ever go to. This can of course be extremely intrusive. Having a database that shows that you visited Google may not matter too much, but how about if you go to a depression counselling site? Abortion advice? Marriage guidance? This can reveal a lot about yourself, information that most of us do not want to see logged and potentially then made available for study. The case has not been made for web-logs to be retained, and they should not be.

The other big issue this bill has to tackle is how to control the broad activities of GCHQ and the others. Before Snowden’s revelations, we were unaware of the hugely broad powers that had been asserted to collect information on every one of us. The intelligence and security services routinely deny that this amounts to mass surveillance, but it is pretty tricky to see that argument, given that they accept that they collect a large amount of information on every one of us, and process it.

The powers they use were mostly not listed in legislation, but somehow inferred between the lines. Words and phrases in the law have been stretched as they were never intended to be used. And if they mean what they say, they should have no objections whatsoever to the legislation expressly banning mass surveillance.

We do of course benefit from the work of GCHQ and others, but to properly support them, we should be clear and transparent about which powers we give them, and how they must be overseen and how they can be approved. This bill provides a chance to set out in one place our rules for the agencies to follow. This will then help them to do their work to keep us safe, as no longer will they have to try to double-guess intentions, and no longer will they be nervous about not doing things because they don’t know if they are allowed.

Across this whole bill, there is also a need for greater protections for people who have a legitimate need for sensitive conversations: journalists, lawyers, politicians, and others. Any of these can of course be involved in criminal behaviour, and so it would be odd to have a blanket ban in place, but there should be a tough threshold to cross to monitor a journalist’s emails from their source, or a lawyer’s with their client.

As for politicians, surely we need to be extremely cautious indeed before allowing the home secretary to be able to put the shadow home secretary under surveillance. While at the moment we can have high confidence that neither the home secretary nor the agency directors would allow this to happen without good cause, we should not legislate hoping the situation will never change.

There is much more that will come up in the bill – I expect a return of the so-called request filter, which makes it much easier to search multiple remote databases at once, and so makes fishing expeditions much easier – and all of us will need to be alert for the details in the legislation.

Other countries are watching what we do.

We also need to beef up our oversight mechanisms. The ones we currently have are inadequate to the task, because they are too fragmented and too poorly resourced. We should have a principle of transparency reporting, so that information is provided, anonymised and aggregated as needed, for us to find out how these powers are being used by the state. And we should inform people when they have been monitored after the monitoring is over, and when it is safe to do so, so that they can seek redress if it was done inappropriately.

Importantly, this bill matters not just for the UK, but around the world. Other countries are watching what we do, and many will copy it. If we decide that we have a legal right to demand information from foreign companies in the USA, so too will China and Russia claim that right. If we defend the idea that it should be up to ministers to approve interception, rather than through judges, the Bahraini government will expand its surveillance of its citizens.

Whether we engage in the debate, and the steps that we take to ensure that only the correct powers are allowed, will shape the form of state surveillance across the world for decades to come.