Last week, Internet traffic for more than two hundred content delivery networks (CDN) and cloud hosting providers was redirected through a Russian state-owned telecommunications provider Rostelcom (AS12389).
The incident was a large scale BGP hijack that affected over 8,700 internet traffic routes from 200+ networks including CDNs and cloud hosting providers such as, Google, AWS, Akamai, Linode, Digital Ocean, Facebook, Cloudflare, GoDaddy, LeaseWeb, Joyent, and Hetzner. It was reported that the incident lasted for about an hour.
How was this BGP routing error by Rostelcom’s network engineers happened:
Earlier this week there was a large scale BGP hijack incident involving AS12389 (Rostelecom) affecting over 8,000 prefixes.— BGPmon.net (@bgpmon) April 5, 2020
Many examples were just posted on @bgpstream , see for example this example for @Facebook https://t.co/Bvzn5PNyFp pic.twitter.com/6aEzFyIfCv
According to the BGPmon.net, a monitoring service owned by Cisco, detected a BGP hijack at 2020-04-01 19:27:28, with the prefix 18.104.22.168/19, which is normally announced by AS32934 FACEBOOK, US. Instead, a more specific route (22.214.171.124/24) was announced by ASN 12389 that was detected by 135 BGPMon peers.
ASN is an autonomous system number through which internet entities are identified.
A security firm Qrator Labs also monitored this BGP leak in real-time and said, “Before the issue was resolved, paths between the largest cloud networks were somewhat disrupted — the Internet blinked. The route leak was distributed quite well through Rascom (AS20764), then Cogent (AS174) and in a couple of minutes through Level3 (AS3356) to the world. The issue suddenly became bad enough that it saturated the route decision-making process for a few Tier-1 ISPs”.
“Given the simplicity of the BGP mistakes, during the coronavirus crisis, it’s so easy to allow for an error. However, with the monitoring data provided, the incident came to an end rather quickly, and the proper routing was restored“, Qrator Labs concluded.
In May 2017, the Russian telco Rostelcom was involved in BGP hacking of the financial giants – Visa, MasterCard, HSBC, and many others.
Similar incidents happened in November 2018, where an ISP Telstra took down a sizable part of the Internet in Australia. Also, a small Nigerian ISP MainOne that affected a few hundred of Google networks.
We have investigated the advertisement of @Google prefixes through one of our upstream partners. This was an error during a planned network upgrade due to a misconfiguration on our BGP filters. The error was corrected within 74mins & processes put in place to avoid reoccurrence— MainOne (@Mainoneservice) November 13, 2018