Yesterday, we published a blog post sharing the news and some initial details about Amazon’s DNS route hijack event to steal Ethereum cryptocurrency from myetherwallet.com. In this post, we’ll explore more details about the incident from the BGP hijack’s perspective.
As noted by Dyn, CloudFlare, and various other entities who monitor Internet routing and health, Amazon’s Route 53 (the DNS service offered by AWS) prefixes were hijacked. A BGP update taken from Isolario suggests that on 24 April, its BGP feeders were correctly receiving 205.251.192.0/23, 205.251.194.0/23, 205.251.196.0/23, 205.251.198.0/23, originated from Amazon (AS16509), until 11:04:00 (UTC). But, at 11:05:41 (UTC), Isolario recorded the first more specific /24 malicious announcements via BGP feeder and the announcements originated from eNET (AS10297) to its peer 1&1 Internet SE (AS8560).
RIPE Stats collected the first more specific malicious advertisement at 11:05:42 (UTC) originating from eNET (AS10297), but this time through peer Hurricane Electric (AS6939).
Exactly at the same time, 11:05:42 (UTC), the Isolario BGP feeder received another update originating from eNET (AS10297) and it was also coming via Hurricane Electric (AS6939).
Hurricane Electric has a worldwide presence and as per its own website, it has over 20,000 BGP sessions with over 7,000 different networks via over 180 major exchange points and thousands of customer and private peering ports. Unfortunately, these malicious advertisements were shared with many of their BGP peers, causing a massive re-routing of Amazon’s Route 53 DNS traffic towards a malicious DNS server hosted in the Equinix Chicago IBX. According to a statement from Equinix (as mentioned in blog posts here and here), “The server used in this incident was not an Equinix server but rather customer equipment deployed at one of our Chicago IBX data centres.” Equinix is merely the Data Centre provider, but they also provide Internet Exchange Points; Equinix has more than 25 Internet Exchange Points around the globe. One of the Equinix IXPs is situated in the Chicago IBX.
Interestingly, at 11:05:44, Isolario recorded another advertisement originating from eNET (AS10297) via a different peer i.e. Shaw Communications Inc. (AS6327). Click to enlarge image.
eNET (AS10297) is peering at the Equinix Chicago IX Route Server and must have shared the malicious advertisement; it is evident that Equinix Chicago IX does filter prefixes, otherwise we would have seen a bigger propagation of these advertisements. More likely, eNET (AS10297) has bilateral peering with Hurricane Electric (AS6939), 1&1 Internet SE (AS8560), and Shaw Communications Inc. (AS6327) because all three of them are present at Equinix Chicago IX.
RIPE Stats also recorded a malicious announcement from eNET (AS10297) at 11:05:54 (UTC) via peer BroadbandOne/WV Fibre (AS19151), which is also peering at Equinix Chicago IX.
This BGP hijack saga lasted almost two hours, and both RIPE Stats and Isolario BGP Feeders started seeing malicious specific prefixes withdrawing from the routing table around 12:55 (UTC).
This problem could have been easily avoided if Hurricane Electric (AS6939), 1&1 Internet SE (AS8560), Shaw Communications Inc. (AS6327), and BroadbandOne/WV Fibre (AS19151) had prefix filtering in place. Thankfully, Level3 (AS3356), Cogentco (AS174), and NTT (AS2914) are all MANRS members and had prefix filters in place, or the damage would have been much bigger. As per Dyn they recorded only 15% of their nodes received malicious specific advertisement originated from AS10297, while NLNOG-RING (AS199036) were getting 87 unique paths to 205.251.192.0/23 (one of the Route53 prefix) originating from Amazon (AS16509) at 10am UTC. But when the attack started at 11:05 UTC they installed 15 new paths for 205.251.192.0/24 (one of the malicious more specific prefix) originated from eNET (AS10297). Out of those 15 unique paths, 12 of them were coming via Hurricane Electric (AS6939).
The reason to hijack Amazon’s Route 53 prefixes was to hijack the DNS itself; details of that is further explained in blog posts by global DNS providers such as CloudFlare, Dyn and Quad9.
As discussed in the previous blog, MANRS can be part of the solution here. Mutually Agreed Norms for Routing Security (MANRS) calls for four simple, but concrete actions ALL network operators should take to reduce the most common routing threats. The first is filtering, which prevents the propagation of incorrect routing information. (The others are anti-spoofing, address validation, and global coordination.) If all the operators along the path had implemented the MANRS actions – especially filtering – this would not have propagated across the Internet like it did.
So, once again. What are you waiting for? Be part of the solution and help protect the core. Join MANRS.