I was kindly notified by the internet security gods that we have some vulnerable NTP servers on our network. This came as a surprise to me as I was not aware of any internet facing NTP servers on my network. This was easily over come as the kind internet gods supplied me with an IP address of the vulnerable server. Low and be hold there it was… An internet facing NTP server.
What was it you say ? Well it was a Cisco 3750x Switch.
Why would you set up a switch as an NTP server that is internet facing with no security ? Well the simple answer is, I didn’t! It did it, itself.
Something that I only just become aware of recently is that Cisco devices will automagically become NTP servers once they have made a successful NTP association with another valid NTP server.
Why is this important ? This is why. http://www.cvedetails.com/cve/CVE-2013-5211/
This exploit allowed attackers to pump allegedly 400Gbps to a CDN service provider in a massive DDoS attack. Yes 400Gbps. Good luck getting finance to sign off a 450Gbps link to over come that.
Right so how do I patch mine ?
Its surprisingly easy to fix once you get your head around the cisco documentation.
IOS router defines the following four types of access for NTP:
1) Peer – permits router to respond to NTP requests and accept NTP updates. NTP control queries are also accepted. This is the only class which allows a router to be synchronized by other devices.
2) Serve – permits router to reply to NTP requests, but rejects NTP updates (e.g. replies from a server or update packets from a peer). Control queries are also permitted.
3) Serve-only – permits router to respond to NTP requests only. Rejects attempt to synchronize local system time, and does not access control queries.
4) Query-only – only accepts NTP control queries. No response to NTP requests are sent, and no local system time synchronization with remote system is permitted.
ntp source Vlan70 # The interface that the packets will be sourced from.
ntp server 192.168.19.51 # NTP server to sync to.
ntp access-group <type> <access-list>
ntp access-group peer 1 # This is the access list that contains the NTP servers that the device will sync to
ntp access-group serve-only 2 # This access list contains the IP addresses that are allowed to query the device for time
access-list 1 permit 192.168.19.51
access-list 2 deny any # Deny all devices that want to sync with me
ntp source Vlan70
ntp access-group peer 1
ntp access-group serve-only 2
ntp server 192.168.19.51
access-list 1 permit 192.168.19.51
access-list 2 deny any
The above configuration will allow your switch to sync with the servers in access list 1, provided that you have added them as NTP servers. It will not however respond to NTP queries from other machines (access-list 2).
Some other considerations with regards to NTP on your switches.
1) Switches translate the NTP DNS name to an IP address at boot, or when you put in the configuration. This means that should the IP address change the switch will no longer be able to sync until the next time it boots or the configuration is reset. This poses a problem when using inter based NTP servers that are out of your control. Suggestions would be to install a NTP server on something like a Raspberry Pi and tell it to rather sync to an internet based NTP server. Then have your switches sync to that device. The NTP service under linux is much more configurable than the switches. Also the pi can be powered by the USB port on the switches.
2) Switches will not sync with stratum 1 devices by default. Stratum 1 devices are devices that believe and report that they are time lords. They do not sync to other devices. These legitimately include atomic clocks and GPS devices that sync with satellites. The switch will check if the peering partner has in fact synced to another server before it will accept it as a time source.
3) Peering takes long! NTP IS SLOW! It may be all about time but its is slower than paint drying!
I have seen NTP take will over 15 minutes to sync to a known working NTP server. Debug messages and slow brewed coffee are your friends here!
Lastly a thought that brought all of this to light.
As quoted from Matthew Prince, CEO, who said that it’s “the start of ugly things to come” because “someone’s got a big, new cannon.”
The internet was invented for EVERYONE. I’m sure it was not invented to be a weapon!