Saturday, December 20. 2014
Update: This blogpost was written before NTS was available, and the information is outdated. If you are looking for a modern solution, I recommend using software and a time server with Network Time Security, as specified in RFC 8915.
tl;dr Several severe vulnerabilities have been found in the time setting software NTP. The Network Time Protocol is not secure anyway due to the lack of a secure authentication mechanism. Better use tlsdate.
Today several severe vulnerabilities in the NTP software were published. On Linux and other Unix systems running the NTP daemon is widespread, so this will likely cause some havoc. I wanted to take this opportunity to argue that I think that NTP has to die.
In the old times before we had the Internet our computers already had an internal clock. It was just up to us to make sure it shows the correct time. These days we have something much more convenient – and less secure. We can set our clocks through the Internet from time servers. This is usually done with NTP.
NTP is pretty old, it was developed in the 80s, Wikipedia says it's one of the oldest Internet protocols in use. The standard NTP protocol has no cryptography (that wasn't really common in the 80s). Anyone can tamper with your NTP requests and send you a wrong time. Is this a problem? It turns out it is. Modern TLS connections increasingly rely on the system time as a part of security concepts. This includes certificate expiration, OCSP revocation checks, HSTS and HPKP. All of these have security considerations that in one way or another expect the time of your system to be correct.
Practical attack against HSTS on Ubuntu
At the Black Hat Europe conference last October in Amsterdam there was a talk presenting a pretty neat attack against HSTS (the background paper is here, unfortunately there seems to be no video of the talk). HSTS is a protocol to prevent so-called SSL-Stripping-Attacks. What does that mean? In many cases a user goes to a web page without specifying the protocol, e. g. he might just type www.example.com in his browser or follow a link from another unencrypted page. To avoid attacks here a web page can signal the browser that it wants to be accessed exclusively through HTTPS for a defined amount of time. TLS security is just an example here, there are probably other security mechanisms that in some way rely on time.
Here's the catch: The defined amount of time depends on a correct time source. On some systems manipulating the time is as easy as running a man in the middle attack on NTP. At the Black Hat talk a live attack against an Ubuntu system was presented. He also published his NTP-MitM-tool called Delorean. Some systems don't allow arbitrary time jumps, so there the attack is not that easy. But the bottom line is: The system time can be important for application security, so it needs to be secure. NTP is not.
Now there is an authenticated version of NTP. It is rarely used, but there's another catch: It has been shown to be insecure and nobody has bothered to fix it yet. There is a pre-shared-key mode that is not completely insecure, but that is not really practical for widespread use. So authenticated NTP won't rescue us. The latest versions of Chrome shows warnings in some situations when a highly implausible time is detected. That's a good move, but it's not a replacement for a secure system time.
There is another problem with NTP and that's the fact that it's using UDP. It can be abused for reflection attacks. UDP has no way of checking that the sender address of a network package is the real sender. Therefore one can abuse UDP services to amplify Denial-of-Service-attacks if there are commands that have a larger reply. It was found that NTP has such a command called monlist that has a large amplification factor and it was widely enabled until recently. Amplification is also a big problem for DNS servers, but that's another toppic.
tlsdate can improve security
While there is no secure dedicated time setting protocol, there is an alternative: TLS. A TLS packet contains a timestamp and that can be used to set your system time. This is kind of a hack. You're taking another protocol that happens to contain information about the time. But it works very well, there's a tool called tlsdate together with a timesetting daemon tlsdated written by Jacob Appelbaum.
There are some potential problems to consider with tlsdate, but none of them is even closely as serious as the problems of NTP. Adam Langley mentions here that using TLS for time setting and verifying the TLS certificate with the current system time is a circularity. However this isn't a problem if the existing system time is at least remotely close to the real time. If using tlsdate gets widespread and people add random servers as their time source strange things may happen. Just imagine server operator A thinks server B is a good time source and server operator B thinks server A is a good time source. Unlikely, but could be a problem. tlsdate defaults to the PTB (Physikalisch-Technische Bundesanstalt) as its default time source, that's an organization running atomic clocks in Germany. I hope they set their server time from the atomic clocks, then everything is fine. Also an issue is that you're delegating your trust to a server operator. Depending on what your attack scenario is that might be a problem. However it is a huge improvement trusting one time source compared to having a completely insecure time source.
So the conclusion is obvious: NTP is insecure, you shouldn't use it. You should use tlsdate instead. Operating systems should replace ntpd or other NTP-based solutions with tlsdated (ChromeOS already does).
(I should point out that the authentication problems have nothing to do with the current vulnerabilities. These are buffer overflows and this can happen in every piece of software. Tlsdate seems pretty secure, it uses seccomp to make exploitability harder. But of course tlsdate can have security vulnerabilities, too.)
Update: Accuracy and TLS 1.3
This blog entry got much more publicity than I expected, I'd like to add a few comments on some feedback I got.
A number of people mentioned the lack of accuracy provided by tlsdate. The TLS timestamp is in seconds, adding some network latency you'll get a worst case inaccuracy of around 1 second, certainly less than two seconds. I can see that this is a problem for some special cases, however it's probably safe to say that for most average use cases an inaccuracy of less than two seconds does not matter. I'd prefer if we had a protocol that is both safe and as accurate as possible, but we don't. I think choosing the secure one is the better default choice.
Then some people pointed out that the timestamp of TLS will likely be removed in TLS 1.3. From a TLS perspective this makes sense. There are already TLS users that randomize the timestamp to avoid leaking the system time (e. g. tor). One of the biggest problems in TLS is that it is too complex so I think every change to remove unneccesary data is good.
For tlsdate this means very little in the short term. We're still struggling to get people to start using TLS 1.2. It will take a very long time until we can fully switch to TLS 1.3 (which will still take some time till it's ready). So for at least a couple of years tlsdate can be used with TLS 1.2.
I think both are valid points and they show that in the long term a better protocol would be desirable. Something like NTP, but with secure authentication. It should be possible to get both: Accuracy and security. With existing protocols and software we can only have either of these - and as said, I'd choose security by default.
I finally wanted to mention that the Linux Foundation is sponsoring some work to create a better NTP implementation and some code was just published. However it seems right now adding authentication to the NTP protocol is not part of their plans.
OpenBSD just came up with a pretty nice solution that combines the security of HTTPS and the accuracy of NTP by using an HTTPS connection to define boundaries for NTP timesetting.
Posted by Hanno Böck in Computer culture, English, Gentoo, Linux, Security at 00:47 | Comments (14) | Trackbacks (3)
Related entries by tags:
Tracked: Dec 22, 18:10
What the GHOST tells us about free software vulnerability management
On Tuesday details about the security vulnerability GHOST in Glibc were published by the company Qualys. When severe security vulnerabilities hit the news I always like to take this as a chance to learn what can be improved and how to avoid similar incide
Weblog: Hanno's blog
Tracked: Jan 30, 00:52
In Search of a Secure Time Source
All our computers and smartphones have an internal clock and need to know the current time. As configuring the time manually is annoying it's common to set the time via Internet services. What tends to get forgotten is that a reasonably accurate clock is
Weblog: Hanno's blog
Tracked: Sep 07, 17:08
Display comments as (Linear | Threaded)
The possibly circular nature of syncing you mentioned is only part of the problem. If this becomes widespread (and somehow avoids circularity), server admins _will_ send bad time stamps to people, just for the lulz.
What's worse: tlsdate has no sense of strata, which makes it impossible to know what quality a given time source has.
On top of that, NTP-the-protocol has measures to mathematically address lag in the connection to the server.
Sorry, tlsdate may be a quick and dirty tool, but it is _not_ a replacement for NTP.
If anything, NTP needs a TCP-and-TLS layer. Yes, this introduces the same CA problem as for all other TLS protocols, but tlsdate has the same problem.
Also, NTP clients are able to use several sources (and they usually do) and determine the best one which makes difficult to just provide a flawed NTP server.
They also keep the clock correctly synchronized by measuring the drift (so it is kept correctly synchronized even without network access for an extended period of time).
At least, they are able to skew the clock instead of just jumping randomly to the correct time. This is important to many programs that are not able to deal correctly with the time going backward or with the time going forward too fast (for example, it could miss a timer).
So I tried it here at a amd64-hardened Gentoo and got (1st message ois harmless, but then ....) :
Dec 20 16:27:50 t44 kernel: grsec: time set by /usr/bin/tlsdate-helper[tlsdate-helper:4227] uid/euid:0/0 gid/egid:0/0, parent /lib64/rc/sh/runscript.sh[runscript.sh:4214] uid/euid:0/0 gid/egid:0/0
Dec 20 18:10:43 t44 tlsdated: [dbus] failed to get name: Connection ":1.40" is not allowed to own the service "org.torproject.tlsdate" due to security policies in the configuration file
Dec 20 18:10:43 t44 tlsdated: Failed to initialize DBus
Dec 20 18:10:43 t44 tlsdated: tlsdated clean up finished; exiting!
Haven't seen an error like this before, but can you please submit this to the Gentoo bug tracker? I think this is not the right place to discuss potential issues in the Gentoo package (but it fails for me, too - another bug, already in the tracker).
Your suggestion is based on the idea that previou solution is insecure, yet the proposed solution is an abuse of other protocols that relays trust without a control mechanism to prove or disprove the values served.
This suggestion would act as a reason not to build something that actually work and would serve only to make people lazy.
Yes, do let NTP die. No, don't replace it with a hack.
how to replace ntpd with tlsdated on ubuntu? only install tlsdate? also uninstall ntpdate? will then system update automaticcally?
apt-get remove ntp; apt-get install tlsdate
tlsdated will automatically be started on system boot.
It seens the author doesn't know how to configure NTP and misses some very basic concepts about its working.
It would help a lot of you could provide any evidence for your claims. What concept am I missing?
Well, as others already said, the lack of strata, the fact that usually one relies on significantly more than 1 ntp timesource and compensation for roundtrip make tlsdate AT BEST a simple, approximative replacement for client-type endnodes. On top of that, promoting a program that in its default config indiscriminately abuses (ab-use, as in hitting a system repeatedly for other purposes than its intended use) can only end with trouble. NTP has over the years also seen its share of abuse, there's even a wikipedia page dedicated to the most notable cases. http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse I understand your concerns, but this article and its indiscriminate promotion of tlsdate doesn't really help.
Ntpdate needs fixing and a TLS layer, no doubt.
But you suggest to just throw it and replace it with a quick and dirty hack, which must be a joke.
Let's build upon what's existing and reasonably solid instead of always bashing the past, throwing it all and restarting from scratch.
It is a loss of time and energy. On top of that, tlsdate functionalities are obviously not even remotely on par with those of NTP. Starting with the architecture (no stratum).
It is simply a quick and dirty hack that may work for a single server, but in no way will have a chance to show up in a datacenter (unless the administrator is a fool).
Of course, the PTB uses NTP to sync the servers that connect directly to those atomic clocks. And so does NASA. And the US Navy. And the US Air Force -- who incidentally controls the constellation of GPS satellites that also need good time service. You didn't think that they would be stupid enough to connect an atomic clock to the unfiltered Internet, did you?
So, you wouldn't be eliminating NTP, you would just add an additional dependency on top of it, and the above discussion isn't even the tip of the iceberg on the problems in it. Maybe we need to let tlsdate age for another 30 years before we can allow it to be considered sufficiently well-tested? ;)
Seriously, I believe that the biggest problem with NTP is much more of an implementation/operational issue. People set up insecure configurations all the time, and they set up configurations that are easily abused. And this isn't restricted to individual people, groups like Red Hat, Debian, D-Link, Netgear, etc... have done the same. Let's fix the configuration problem before we declare the protocol itself to be heinous and in desperate need of replacement.
Don't get me wrong, there are plenty of areas that can be improved in NTP-the-protocol. But there is a reason why it has lasted for decades. And the weaknesses in the Reference Implementation from ntp.org are not necessarily weaknesses in the protocol. And the configuration weaknesses are not necessarily weaknesses in the Reference Implementation from ntp.org.
There are efforts underway to better integrate with IEEE 1588 PTP, Poul Henning-Kamp has been working on an independent re-implementation from the ground up, and many others. If you want to make your time service secure, I would suggest that maybe you want to get involved in some of these efforts.
Otherwise, well, how much did you pay for your time service?
"The TLS timestamp is in seconds, adding some network latency you'll get a worst case inaccuracy of around 1 second, certainly less than two seconds."
I could see a hybrid approach. Use TLS to set time to within one second and NTP to get it the rest of the way. If NTP comes in outside of the time that TLS reports it is rejected.
This hybrid approach is what OpenNTPD now does:
Problem is: it depends on LibreSSS's libtls API, so it's not available on many systems.
You can find my web page with links to my work as a journalist at https://hboeck.de/.
You may also find my newsletter about climate change and decarbonization technologies interesting.
Hanno on Mastodon
Show tagged entries