There’s a lot of panic going around on the internet today, about something called the Heartbleed bug. I’ve gotten questions, so I’m giving answers.
I’ve heard lots of hype. Is this really a big deal?
Damn right it is!
It’s pretty hard to wrap your head around just how bad this actually is. It’s probably even more of a big deal that the hype has made it out to be!
This bug affects around 90% of all sites on the internet that use secure connections. Seriously: if you’re using the internet, you’re affected by this. It doesn’t matter how reputable or secure the sites you connect to have been in the past: the majority of them are probably vulnerable to this, and some number of them have, in all likelihood, been compromised! Pretty much any website running on linux or netbst, using Apache or NGINX as its webserver is vulnerable. That means just about every major site on the net.
The problem is a bug in a commonly used version of SSL/TLS. So, before I explain what the bug is, I’ll run through a quick background.
What is SSL/TLS?
When you’re using the internet in the simplest mode, you’re using a simple set of communication protocols called TCP/IP. In basic TCP/IP protocols, when you connect to another computer on the network, the data that gets sent back and forth is not encrypted or obscured – it’s just sent in the clear. That means that it’s easy for anyone who’s on the same network cable as you to look at your connection, and see the data.
For lots of things, that’s fine. For example, if you want to read this blog, there’s nothing confidential about it. Everyone who reads the blog sees the same content. No one is going to see anything private.
But for a lot of other things, that’s not true. You probably don’t want someone to be able to see your email. You definitely don’t want anyone else to be able to see the credit card number you use to order things from Amazon!
To protect communications, there’s another protocol called SSL, the Secure Sockets Layer. When you connect to another site that’s got a URL starting with https:, the two computers establish an encrypted connection. Once an SSL connection is established between two computers, all communication between them is encrypted.
Actually, on most modern systems, you’re not really using SSL. You’re using a successor to the original SSL protocol called TLS, which stands for transport layer security. Pretty much everyone is now using TLS, but many people still just say SSL, and in fact the most commonly used implementation of it is in package called OpenSSL.
So SSL/TLS is the basic protocol that we use on the internet for secure communications. If you use SSL/TLS correctly, then the information that you send and receive can only be accessed by you and the computer that you’re talking to.
Note the qualifier: if you use SSL correctly!
SSL is built on public key cryptography. What that means is that a website identifies itself using a pair of keys. There’s one key, called a public key, that it gives away to everyone; and there’s a second key, called a private key, that it keeps secret. Anything that you encrypt with the public key can only be decrypted using the private key; anything encrypted with the private key can only be decrypted using the public key. That means that if you get a message that can be decrypted using the sites public key, you know that no one except the site could have encrypted it! And if you use the public key to encrypt something, you know that no one except that site will be able to decrypt it.
Public key cryptography is an absolutely brilliant idea. But it relies on the fact that the private key is absolutely private! If anyone else can get a copy of the private key, then all bets are off: you can no longer rely on anything about that key. You couldn’t be sure that messages came from the right source; and you couldn’t be sure that your messages could only be read by an authorized person.
So what’s the bug?
The SSL protocol includes something called a heartbeat. It’s a periodic exchange between the two sides of a connection, to let them know that the other side is still alive and listening on the connection.
One of the options in the heartbeat is an echo request, which is illustrated below. Computer A wants to know if B is listening. So A sends a message to B saying “Here’s X bytes of data. Send them back to me.” Then A waits. If it gets a message back from B containing the same X bytes of data, it knows B was listening. That’s all there is to it: the heartbeat is just a simple way to check that the other side is actually listening to what you say.
The bug is really, really simple. The attacker sends a heartbeat message saying “I’m gonna send you a chunk of data containing 64000 bytes”, but then the data only contains one byte.
If the code worked correctly, it would say “Oops, invalid request: you said you were going to send me 64000 bytes of data, but you only sent me one!” But what the buggy version of SSL does is send you that 1 byte, plus 63,999 bytes of whatever happens to be in memory next to wherever it saved the byte..
You can’t choose what data you’re going to get in response. It’ll just be a bunch of whatever happened to be in memory. But you can do it repeatedly, and get lots of different memory chunks. If you know how the SSL implementation works, you can scan those memory chunks, and look for particular data structures – like private keys, or cookies. Given a lot of time, and the ability to connect multiple times and send multiple heartbeat requests each time you connect, you can gather a lot of data. Most of it will be crap, but some of it will be the valuable stuff.
To make matters worse, the heartbeat is treated as something very low-level which happens very frequently, and which doesn’t transfer meaningful data. So the implementation doesn’t log heartbeats at all. So there’s no way of even identifying which connections to a server have been exploiting this. So a site that’s running one of the buggy versions of OpenSSL has no way of knowing whether or not they’ve been the target of this attack!
See what I mean about it being a big deal?
Why is it so widespread?
When I’ve written about security in the past, one of the things that I’ve said repeatedly is: if you’re thinking about writing your own implementation of a security protocol, STOP. Don’t do it! There are a thousand ways that you can make a tiny, trivial mistake which completely compromises the security of your code. It’s not a matter of whether you’re smart or not; it’s just a simple statement of fact. If you try to do it, you will screw something up. There are just so many subtle ways to get things wrong: it takes a whole team of experts to even have a chance to get it right.
Most engineers who build stuff for the internet understand that, and don’t write their own cryptosystems or cryptographic protocols. What they do is use a common, well-known, public system. They know the system was implemented by experts; they know that there are a lot of responsible, smart people who are doing their best to find and fix any problems that crop up.
Imagine that you’re an engineer picking an implementation of SSL. You know that you want as many people trying to find problems as possible. So which one will you choose? The one that most people are using! Because that’s the one that has the most people working to make sure it doesn’t have any problems, or to fix any problems that get found as quickly as possible.
The most widely used version of SSL is an open-source software package called OpenSSL. And that’s exactly where the most bug is: in OpenSSL.
How can it be fixed?
Normally, something like this would be bad. But you’d be able to just update the implementation to a new version without the bug, and that would be the end of the problem. But this case is pretty much the worst possible case: fixing the implementation doesn’t entirely fix the problem. Because even after you’ve fixed the SSL implementation, if someone got hold of your private key, they still have it. And there’s no way to know if anyone stole your key!
To fix this, then, you need to do much more than just update the SSL implementation. You need to cancel all of your keys, and replace them new ones, and you need to get everyone who has a copy of your public key to throw it away and stop using it.
So basically, at the moment, every keypair for nearly every major website in the world that was valid yesterday can no longer be trusted.
Unless I’m misunderstanding this, this is effectively the same thing as a buffer-overflow bug, except instead of writing to out-of-bounds memory, it’s reading from it. That is, the fundamental problem and the ways to protect against it were known 50 years ago. It’s fairly exhausting to see same bugs with known solutions over and over again.
The bug may be even stupider than that! A classic fail is assuming an operation was successful and not thoroughly cross-checking the results. For example, doing some kind of read operation where you already (supposedly) know how many octets you received and assuming a successful read implies you got all the data.
I keep this quote close on hand: “A good programmer is someone who looks both ways before crossing a one-way street.” – Doug Linder
That’s a great quote! The really great ones look up, too! 🙂
It’s a fundamental awareness of possible unexpected conditions that’s necessary for writing robust code. Sure, it’s a one-way street, but cars drive on streets, and people sometimes make a wrong turn, so it’s possible a car could be coming the wrong way. So you’d damn well better check!
This is the third serious flaw in security software in the last two months (Apple and GNU TLS are the others).
All of them occurred in code written in C and the cause of the defect was due to a known common failure mode of C programming.
I am starting to think that no matter how smart you are, you are not smart enough to code in C.
Pingback: Heartbleed | Ok, panico
So, in terms of “You need to cancel all of your keys, and replace them new ones, and you need to get everyone who has a copy of your public key to throw it away and stop using it.” how exactly do I do this? Do I simply create new passwords for everything with a https? Or do I need to dig into the Options menu of my browser?
Who he means are the admins. The admins of a site need to do this. You actually can’t do this in oyur browser (not that I know of).
But yes, you should change your passwords.
The relevant keypairs are controlled by the people who run the websites. They need to discard their keys, and stop accepting requests from people that use cached keys. There’s nothing that you can do – from your point of view, it should all be automatic.
In terms of passwords: yeah, you should probably change them. Your passwords could have been grabbed. It’s unlikely, but it’s possible, and better safe than sorry.
I know just enough about programming to make it seem like this is a very stupid error. How do you not check such a thing? How do you go so long with nobody noticing?
Welcome to the new world. The social attitudes of immediate and visceral have replaced the long and thoughtful, and the “it’s all good” casualness pervades. This attitude seems to infect too many workers today.
There’s a sign that often hangs in an Engineer’s office: “You can have fast, cheap or good: pick two.” Increasingly managers see a requirement for fast and cheap, so quality is sacrificed.
One would like to believe that, when working on a security protocol that quality would be “job number one,” but that’s not a belief one can count on anymore.
That is a generalisation that’s lazier and more idiotic than you accuse the programmers of being. Security programming is _hard_, and it takes a team of experts to have even a _tiny chance_ to do it right. Screwing up in this way doesn’t indicate lack of attention to quality; it indicates that humans are fallible and C code without memory leaks is really hard to write.
(It possibly also indicates that nothing secure should ever be written in C, but I didn’t see anyone offering to pay for a managed language rewrite before Heartbleed was found.)
There _are_ no managers of OpenSLL, since the number of people paid to work full-time on OpenSSL is _zero_. There’s a core group of about four people, and about another 6-8 volunteer programmers. (Including a consultant Google sort-of lends to the project some of the time.)
For a critical piece of global infrastructure, nobody actually wanted to fund it.
You might want to get to know me better before you call me lazy and idiotic. I was obviously speaking of a general tendency and mood that I see infecting our thinking in many ways and in many places.
And while I am sympathetic to the realities, I stand by the assertion that this was a stupid bug and that a potentially large security breech is due to foolish behavior driven, in part, by stupid current social attitudes.
The idea that any “secure” commercial website would use free code written for free by unknown programmers in their spare time is, if you really think about it, a bit extraordinary. But managers at these sites are driven by the need for the site to make a profit, and so they look for all opportunities to save money. And open source is generally a good bet for the same reason Wikipedia is.
Nevertheless, the open source – commercial website dynamic is one part of the problem. Efforts like OpenSSL absolutely should be funded!! On the flip side, perhaps we should rethink using free software in crucial commercial contexts.
I also maintain that the bug itself is the result of sloppy thinking. The context isn’t just “network” but “secure network!” You’re working in a context where you have to assume highly knowledgeable, highly motivated, explicitly malicious input. Working in a “network” context is a whole level above working in a “user input” context, and a “secure network” context is a whole level above that. (Even worse: a “low-level secure network” context!)
So I think it is unforgivable in a context like that to make any assumption about the input. Checking everything should be a matter of course. So should a second pair of eyes checking everything.
You can find cases where people bewail the rise in crime despite numbers establishing a clear substantial drop in crime, so I look with askance at any claim that today is worse then days past.
We can talk about the the Morris Worm and the not-accidental, “it’s all good”ness of OS and JEDGAR (imagine a multiuser system letting people spy on other people today!). But more importantly, I don’t see any cryptographic security in use across networks prior to 1994. Between 1969 and 1995, if you wanted to log into a system over the net, you used telnet and passed your password and everything you were doing in plaintext. Same thing with FTP–born in 1971, secure alternatives published in the 1990s. SSL was created in 1994. For 25 years, every net protocol could be watched by a simple packet sniffer.
So no, it’s not fair to wave a hand at modern programmers, and act as if security problems were a new thing. Nor is it fair to wave a hand at mangers, either; the prototypical example of fast and cheap for me is E.T. for the Atari 2600, where they gave a programmer 6 weeks to produce a game and made five million copies of it.
I don’t want to abuse our host’s hospitality, so we might want to take this elsewhere. I wrote my own blog post about Heartbleed, and I suggest we pursue it there if you’re inclined to continue this.
Here I’ll just say that, “it was ever thus” is a view that goes back to Ecclesiastes, and it’s often a view with which I agree. But I believe a clear-eyed look at the world today reveals something “new under the sun” — the combination of communications and easy computer access has an impact on the world I think is on par with fire. As just one example, when in history have we needed metal detectors in grade schools? The world is changing in ways we’ve never seen before.
But that’s a complex topic with no easy answers. I find the evidence compelling.
More on topic is that an error involving not validating user input when working in a low-level network security context is inexcusable. There’s no claim here that security problems are new – -that they aren’t is part of what’s inexcusable.
Bottom line: the programmer made a stupid mistake. My guess is that any programmer good enough to be working on such code in the first place would be savvy enough to be the first one to agree that they made a stupid, catastrophic error.
Yes, there’s something new under the sun, but I don’t see any evidence that security has got laxer because of “stupid current social attitudes”.
Metal detectors mean nothing. You can’t prove that a problem has got worse because the response to it has changed like that.
Not to be pedantic, but here:
“The SSL protocol includes something called a heartbeat.”
It’s really the TLS protocol that includes the heartbeat. [1]
Actually, yeah. I’m being pedantic 😉
[1] https://tools.ietf.org/html/rfc6520
The implications of bug are actually even worse than what you’ve described, because most web servers don’t support forward secrecy. This means that if an attacker grabs a server’s private key, they can not only use it to impersonate that server (until the key is revoked and replaced), but also go back and decrypt any old sessions that they happen to have recorded (see the EFF’s comment on the subject). And there’s nothing you can do to stop or even detect this.
Mind you, the NSA is the only group I can think of that’s likely to have recorded a lot of encrypted sessions on the off chance they might be able to decrypt them later. And we trust the NSA, right?
BTW, the bug itself is even dumber than David Starner’s comment suggests, because OpenSSL actually bypasses any overflow protection in the standard memory allocation system. Theo de Raadt (of OpenBSD, a security-oriented branch of unix) explains here, here, and here.
There are web sites listing affected sites and unaffected sites (which are OK as long as your password isn’t also used on an affected site).
http://mashable.com/2014/04/09/heartbleed-bug-websites-affected/#:eyJzIjoiZiIsImkiOiJfNnowMnc0ancxcjRuaWI4dSJ9
http://mashable.com/2014/04/09/heartbleed-bug-websites-affected/#:eyJzIjoiZiIsImkiOiJfNnowMnc0ancxcjRuaWI4dSJ9
Whoops,
http://www.cnet.com/how-to/which-sites-have-patched-the-heartbleed-bug/
And of course there are rumors that the NSA has known for years.
Pingback: Heartbleed bug | The Hard-Core Coder