The Investigatory Powers Bill for architects and administrators

OK, it’s not the end of the world. But it does change things radically, should it pass third reading in its current form. There is, right now, an opportunity to effect some change to the bill in committee stage, and I urge you to read it and the excellent briefings from Liberty and the Open Rights Group and others and to write to your MP.

Anyway. What does this change in our threat models and security assessments? What aspects of security validation and testing do we need to take more seriously? I’m writing this from my perspective, which is from a small ISP systems perspective, but this contains my personal views, not that of my employer, yada yada.

The threats

First up let’s look at what the government can actually do under this bill. I’m going to try and abstract things a little from the text in the bill, but essentially they can:

  • Issue a technical capability notice, which can compel the organization to make technical changes to provide capability to provide a service to government
  • Compel an individual (not necessarily within your organization) to access data
  • Issue a retention notice, which can compel the organization to store data and make it available through some mechanism
  • Covertly undertake equipment interference (potentially with the covert, compelled assistance of someone in the organization, potentially in bulk)

Assuming we’re handling some users’ data, and want to protect their privacy and security as their bits transit the network we operate, what do we now need to consider?

  • We can’t trust any individual internally
  • We can’t trust any small group of individuals fully
  • We can’t trust the entire organization not to become compromised
  • We must assume that we are subject to attempts at equipment interference
  • We must assume that we may be required to retain more data than we need to

So we’re going to end up with a bigger threat surface and more attractors for external threats (all that lovely data). We’ve got to assume individuals may be legally compelled to act against the best interests of the company’s users – this is something any organization has to consider a little bit, but we’ve always been viewing this from a perspective of angry employees the day they quit and so on. We can’t even trust that small groups are not compromised and may either transfer sensitive data or assist in compromise of equipment

Beyond that, we have to consider what happens if an organizational notice is made – what if we’re compelled to retain none-sampled flow data, or perform deep packet inspection and retain things like HTTP headers? How should we defend against all of this, from the perspective of keeping our users safe?


To be clear – I am all for targeted surveillance. I believe strongly we should have well funded, smart people in our intelligence services, and that they should operate in a legal fashion, with clear laws that are up to date and maintained. I accept that no society with functional security services will have perfect privacy.

I don’t think the IPB is the right solution, mind you, but this is all to say that there will always be some need for targeted surveillance and equipment interference. These should be conducted only when a warrant is issued (preferably by a judge and cabinet minister), and ISPs should indeed be legally able to assist in these cases, which requires some loss of security and privacy for those targeted users – and it should be only those users.

I am a paid-up member of the Open Rights Group, Liberty and the Electronic Frontiers Foundation. I also regularly attend industry events in the tech sector and ISP sector in particular. Nobody wants to stop our spies from spying where there’s a clear need for them to do so.

However, as with all engineering, it’s a matter of tradeoffs. Bulk equipment interference or bulk data retention is complete overkill and helps nobody. Covert attacks on UK infrastructure actively weaken our security. So how do we go about building a framework that permits targeted data retention and equipment interference in a secure manner?  Indeed, encourages it at an organizational level rather than forcing it to occur in a covert manner?

Equipment Interference

This is the big one, really. Doesn’t matter how it happens – internally compelled employee, cleaner plugging a USB stick from a vague but menacing government agency into a server and rebooting it, switches having their bootloaders flashed with new firmware as they’re shipped to you, or covert network intrusion. Either way you end up in a situation where what your routers, switches, servers etc are doing things you did not expect, almost certainly without your knowledge.

This makes it practically impossible to ensure they are secure, against any threats. Sure, your Catalyst claims to be running IOS 13.2.1. Your MX-series claims to be running JunOS 15.1. Can we verify this? Maybe. We can use host-based intrusion detection systems to monitor integrity and raise alarms.

Now, proper auditing and logging and monitoring of all these devices, coupled with change management etc will catch most of the mundane approaches – that’s just good infosec, and we have to do that to catch all the criminals, script kiddies and random bots trawling the net for vulnerable hosts. Where it gets interesting is how you protect against the sysadmin themselves.

It feels like we need to start implementing m-in-n authorization to perform tasks around sensitive hosts and services. Some stuff we should be able to lock down quite firmly. Reconfiguring firewalls outside of the managed, audited process for doing so using a configuration management (CM) tool? Clearly no need for this, so why should anyone ever be able to do it? All services in CM, be it Puppet/Salt/Chef, with strongly guarded git and puppet repositories and strong authentication everywhere (keys, proper CA w/ signing for CM client/server auth, etc)? Then why would admins ever need to log into machines? Except inevitably someone does  need to, and they’ll need root to diagnose whatever’s gone wrong, even if the fix is in CM eventually.

We can implement 2-person or even 3-person authentication quite easily, even at small scales, using physical tools – hardware security modules locked in 2-key safes, or similar. But it’s cumbersome and complicated, and doesn’t work for small scales where availability is a concern – is the on-call team now 3 people, and are they all in the office all the time with their keys?

There’s a lot that could be done to improve that situation in low to medium security environments, to stop the simple attacks, to improve the baseline for operational security, and crucially to surface any covert attempts at EI conducted by an individual or from outside, covertly. Organizationally, it’d be best for everyone if the organization were aware of modifications that were required to their equipment.

From a security perspective, a technical capability notice or data retention notice of some sort issued to the company or group of people at least means that a discussion can be had internally. The organization may well be able to assist in minimising collateral damage. Imagine: “GCHQ needs to look at detailed traffic for this list of 10 IPs in an investigation? Okay, stick those subscribers in a separate VLAN once they hit the edge switches, route that through the core here and perform the extra logging here for just that VLAN and they’ve got it! Nobody else gets logged!” rather than “hey, why is this Juniper box suddenly sending a few Mbps from its management interface to some IP in Gloucestershire? And does anyone know why the routing engines both restarted lately?”

Data Retention

This one’s actually pretty easy to think about. If it’s legally compelled by a retention or technical capability notice, you must retain as required, and store it as you would your own browser history – in a write-only secure enclave, with vetted staff, ISO27K1 compliant processes (plus whatever CESG requires), complete and utter segmentation from the rest of the business, and whatever “request filter” the government requires stays in there with dedicated, highly monitored and audited connectivity.

What’s that, you say? The government is not willing to pay for all that? The overhead of such a store for most small ISPs (<100,000 customers) would be huge. We’re talking millions if not more per small ISP ( lists 237 ISPs in the UK). Substantial office space, probably 5 non-technical and 5 technical staff at minimum, a completely separate network, data diodes from the collection systems, collection systems themselves, redundant storage hardware, development and test environments, backups (offsite, of course – to your second highly secure storage facility), processing hardware for the request filter, and so on. Just the collection hardware might be half a million pounds of equipment for a small ISP. If the government start requiring CESG IL3 or higher, the costs keep going up. The code of practice suggests bulk data might just be held at OFFICIAL – SENSITIVE, though, so IL2 might be enough.

The biggest risk to organizations when it comes to data retention is that the government might not cover your costs – they’re certainly not required to. And of course the fact that you’re the one to blame if you don’t secure it properly and it gets leaked. And the fact that every hacker with dreams of identity theft in the universe now wants to hack you so bad, because you’ve just become a wonderfully juicy repository of information. If this info got out, even for a small ISP, and we’re talking personally-identifiable flow information/IP logs – which is what “Internet Connection Records” look/sound like, though they’re still not defined – then Ashley Madison, TalkTalk and every other “big data breach” would look hilariously irrelevant by comparison. Imagine what personal data you could extract from those 10,000 users at that small ISP! Imagine how many people’s personal lives you could utterly destroy, by outing them as gay, trans or HIV positive, or a thousand other things. All it would take is one tiny leak.

You can’t do anything to improve the security/privacy of your end users – at this point, you’re legally not allowed to stop collecting the data. Secure it properly and did I mention you should write to your MP while the IPB is at committee stage?

If you’ve not been served with a notice: carry on, business as usual, retain as little as possible to cover operational needs and secure it well.


Auditing isn’t a thing that happens enough.

I always think that auditing is a huge missed opportunity. We do pair programming and code review in the software world, so why not do terminal session reviews? If X logs into a router, makes 10 changes and logs out, yes we can audit the config changes and do other stateful analysis, but we can audit those commands as a single session. It feels like there’s a tool missing to collate logs from something like syslog and bring them together as a session, and then expose that as a thing people can look over, review, and approve or flag for discussion.

It’s a nice way for people to learn, too – I’ve discovered so many useful tools from watching my colleagues hack away at a server, and about the only way I can make people feel comfortable working with SELinux is to walk them through the quite friendly tools.

Auditing in any case should become a matter of course. Tools like graylog2, ELK and a smorgasbord of others allow you to set up alerts or streams on log lines – start surfacing things like root logins, su/sudo usage, and “high risk” commands like firmware updates, logging configuration, and so on. Stick a display on your dashboards.

Auditing things that don’t produce nice auditable logs is of course more difficult – some firewalls don’t, some appliances don’t. Those just need to be replaced or wrapped in a layer that can be audited. Web interface with no login or command audit trail? Stick it behind a HTTPS proxy that does log, and pull out POSTs. Firewall with no logging capability? Bin it and put something that does in. Come on, it’s 2016.

Technical capability notices and the rest

This is the unfixable. If you get handed a TCN, you basically get to do what it says. You can appeal on the grounds of technical infeasibility, but not proportionality or anything like that. So short of radically decentralizing your infrastructure to make it technically too expensive for the government, you’re kind of stuck with doing what they say.

The law is written well enough to prevent obvious loopholes. If you’re an ISP, you might consider encryption – you could encrypt data at your CPEs, and decrypt it on your edge. You could go a step further and not decrypt it at all, but pass it to some other company you notionally operate extraterritorially, who decrypt it and then send it on its way from there. But these come with potentially huge cost, and in any case the TCN can require you to remove any protection you applied or are in a position to remove if practical.

We can harden infrastructure a little – things like using n-in-m models, DNScrypt for DNS lookups from CPEs, securely authenticating provisioning servers and so on. But there is no technical solution for a policy problem – absolutely any ISP, CSP, or 1-man startup in the UK is as powerless as the next if the government rocks up with a TCN requiring you to store all your customers’ data or to install these black boxes everywhere your aggregation layer connects to the core or whatever.

Effectively, then, the UK industry is powerless to prevent the government from doing whatever the hell it likes, regardless of security or privacy implications, to our networks, hardware and software. We can take some steps to mitigate covert threats or at least give us a better chance of finding them, and we can make some changes which attempt to mitigate against compelled (or hostile) actors internally – there’s an argument that says we should be doing this anyway.

And we can cooperate with properly-scoped targeted warrants. Law enforcement is full of good people, trying to do the right thing. But their views on what the right thing to do is must not dictate political direction and legal implementation while ignoring the technical realities. To do so is to doom the UK to many more years with a legal framework which does not reflect reality, and actively harms the security of millions of end users.