Huawei root access is BAD! VERY, VERY BAD: Or, how we reasoned ourselves out of root access by default

OK, So Trump didn't actually say that about Huawei. But given his recent declaration of all-out war on them, it wouldn't surprise me if he did.

As you probably know, the notorious Chinese tech company was blacklisted by Google this week on the instructions of the Trump administration. All this high-profile paranoia about security has got me thinking about our approach to the subject as we prepare to release v8.3.7 of the load balancer appliance.

When policy causes pain

I've recently spent a fair bit of time empathizing with a customer in pain. He was replacing some kit that complied 100% with his internal security policy, which was very strict because he was on a government dark site. Unfortunately he'd suffered a security breach where this 100% government compliant kit was hacked. So he wanted to replace it with a pair of Loadbalancer.org units.

The irony was that although he particularly liked the fact that he had root access, and could easily check that the open source code was up to the latest security standards, our kit did NOT meet his internal security policy requirements — precisely because he could get that reassuring root access.

Going in circles

Discussions about security are hard to win, partly because they frequently start with circular reasoning.

"Circular reasoning is when you attempt to make an argument by beginning with an assumption that what you are trying to prove is already true."

Some of my favourite examples are:

1. You must have a dedicated firewall — or you can't be secure
2. You must have a WAF — if you can't prove your code is secure
3. You must have a DMZ — or you will get hacked
4. You must not allow root access...

I’ve heard people raise one of these circular arguments followed by, "I can't buy this product because it's not secure".

Open-source software is particularly incompatible with some government-style dogma such as, "It can't be secure if anyone can see the code". Which is based on terrible logic. How can you be secure if you can't see what you are running?
HIPAA, FIPS, FISMA, NIST, PCI-DSS are all guilty of taking sensible rules of thumb about security and turning them into iron-clad rules that can't be worked around (even if they are clearly wrong).

All the major network security vendors ship network appliances based on Linux with full root access available. Why? Because they know two things:

1. Open-source software is more secure.
2. Any half-competent hacker with physical access can rootkit any appliance anyway.

Obfuscation is a terrible defense mechanism, if it's your only defense mechanism.

Perceived threats vs the real deal

Our head of security has been telling me that we need to change our default for a few years now. But I’ve always been sceptical: there are so many scare stories out there, and I’ve only been interested in responding to the real threats, like Heartbleed.

In my defence, I’m confident that any genuine security issue would see half the company working on a fix and rapid release. So it’s frustrating to see false alarms get so much airtime and cost us engineering hours.

We’re a trusted company because our product is secure

Our historical stance has always been to explain the reasoning behind our security logic, followed up with details on how to change the security settings for any desired outcome. The advantage of this being that it makes the customer think very hard about what their security exposure really is.

This worked fine fifteen years ago, when our core customer was a Linux engineer with sole decision-making responsibility, who loved that fact that we offered root access. But now that we’re working with enterprise business partners, many of our customers have very different requirements and limitations.

We’ve changed our product to help customers

All of this brings me to the fundamental changes we’ve made in v8.3.7 of our product to help customers who are bound by strict security policies. These changes have always been possible. But this time we've tried to make it simple by default:

• Root access is disabled
• Execute shell command is disabled
• Manual firewall is disabled
• Secure key management is disabled
• Console access is disabled
• API is disabled
• Unencrypted (HTTP) access to web administration is disabled

You can re-enable these options via the web interface (if you have administration rights). But we recommend you disable them again after making any changes required.

And if you are really brave — you can make the whole thing irrevocable!

On balance we feel this is a sensible step towards achieving our company mission: helping you achieve zero downtime.

So, is root access bad or not?

I recommend that you treat any pivotal network infrastructure appliance as a single bastion host. Only a very small group of people should have administration access, which is reviewed frequently to confirm a continued business need exists. You should probably also integrate the access with your core authorization system (i.e. Active Directory). You should use both an external firewall and the built-in lockdown wizard.

Because if a hacker gets any kind of access to your core systems, let alone root access - you are in trouble. Privilege escalation is one of the most common bugs exploited by hackers on servers with multiple users.

So root access itself is not necessarily BAD, unless anyone BAD gets access. Then it's BAD, VERY, VERY BAD.

How's that for circular reasoning?