Please note, updates to this post can be found further down the page.

It seems that Telia, one of the underwater transatlantic cables which connects the Internet between Europe and the US, has been reportedly cut or damaged. The specific cable believed to have been cut was TAT-14, the route of which can be seen below as it travels from the US through to the UK, France, the Netherlands, Germany and Denmark.

For a large number of Internet users in Europe, this resulted in outages of many popular sites including Dropbox, and sites using CloudFlare, until hosts were able to re-route web traffic via other unaffected routes.

Luckily, there are many other transatlantic cables, so this didn’t take too long, and it was at most an hour or so before sites could be accessed again – although there was considerable congestion afterwards, as can be seen by the below screenshot of Akamai‘s website late on 19th May.

News quickly spread on a number of online forums including:

• http://lowendtalk.com/discussion/27622/telia-s-transatlantic-cable-died
• http://lowendtalk.com/discussion/27622/telia-transatlantic-cable-outage
• https://news.ycombinator.com/item?id=7769587

TheRegister also published an article on this – although fairly late after it happened.

Digital Ocean and CloudFlare seemed to be the two companies mostly affected (who it is assumed have transit links with Telia), but many other websites and web hosts were indirectly affected through customers using an ISP which uses Telia as one of their connections.

Digital Ocean:

@jnisigl @martystronguk we've rerouted so we are not using telia right now. I'm pretty sure they confirmed an atlantic cut. Pretty rare.

— John Edgar (@jedgar) May 19, 2014

CloudFlare:

Upstream networking issues are causing connectivity issues in Eastern Europe which is affecting multiple US and EU PoPs.

— CloudFlareStatus (@CloudFlareSys) May 19, 2014

@gwilhelm One of our upstreams (Telia) appears to have a trans-Atlantic cable cut. We're routing around in many EU locations now.

— CloudFlare (@CloudFlare) May 19, 2014

A number of users also took to Twitter to find out if they were the only ones experiencing problems accessing certain parts of the Internet:

rackspace trans-atlantic connectivity issues. cloudflare connectivity issues in eastern europe. what's next ? linode ? #fuckme

— Carlos Andrade (@karlus) May 19, 2014

Is anybody else in Europe having problems reaching sites in the U.S.? Telia’s network does not work for me right now.

— Roland Kuhn (@rolandkuhn) May 19, 2014

UPDATE (11:45pm GMT – 19th May):

Telia has released a statement via Twitter which explains the cause of the outage as a ‘planned update gone wrong’

Vår tidigare driftstörning på vår internet trafik mot USA och Asien är avhjälpt sen 23-tiden, det var en planerad uppdatering som gick fel

— Telia Kundservice (@Telia_service) May 19, 2014

Which roughly translates from Swedish to:

“Our previous disruption on our internet traffic against the United States and Asia is eliminated since 23 o’clock, it was a planned update went wrong”

UPDATE 2 (12 midnight GMT – 19th May:

Telia have now published an article in Swedish, in which Bo Djurberg (Manager at Telia) summarises the issues they had.

Again, roughly translated this is:

Problems with Telia Internet
Telia had tonight big problems with their Internet services.
Users could not get into websites registered in the United States.
– There was a scheduled job that went a bit wrong, says Bo Djurberg, manager at Telia.
The problems started at 22 o’clock on Monday evening.
Lots of customers had problems with the internet connection. According to Telia these involved a planned update that went wrong.
– They had problems reaching websites in the U.S. and vice versa, says Bo Djurberg.
At 23 o’clock the problems were remedied, according to Telia.
– Now everything will work out. All traffic was down for about an hour, says Djurberg.
Exactly how many users are affected is not clear.

UPDATE 3 (8am GMT – 20th May):

A further statement by Telia says that the issue was caused by a software ‘route change’ rather than the physical cable being cut. This explains how Telia were able to fix the problem so soon – something which likely wouldn’t have been so soon had it been cut.

UPDATE 4 (10am GMT – 20th May):

Further confirmation of this being a ‘planned update’

@achimsawall It was a planned update that unfortunately went wrong. We're sorry for that but it should be fixed now. /CT

— Telia Kundservice (@Telia_service) May 20, 2014

This post will be updated as and when more information is released.

UPDATE 5 (4pm GMT – 20th May):

This will likely be the last update, so I just wanted to summarise all of the updates and new developments since this post was first published.

This morning, Telia announced that the disruption was due to a ‘planned update went wrong’. There was no physical damage or cut to any of the cables, so it seems they were doing some form of software update for the cable (which we believe was TAT-14), which didn’t go as planned and resulted in the cable failing to carry data and going out of service.

Thankfully the downtime last night didn’t last too long as ISPs and web hosts were quick to re-route traffic via alternate transit carriers. There are many other transatlantic fibre cables, so this wouldn’t have been to hard – although congestion was caused as traffic had to be funneled to these other routes.

Were you affected by the Telia issues? Please get in touch (adam[at]adamowen.co.uk), or add a comment and I’ll feature your quote.

I also wrote another article for my company here – http://www.spiralmedia.co.uk/slow-internet-last-night.