Router configuration changes affected tools and systems employees used in day-to-day operations
Facebook revealed in an official blog post on Monday evening that the six-hour outage across its platforms including Instagram, Messenger, Whatsapp, and OculusVR were caused by configuration changes to its routers.
Shortly before 5pm on Monday, the three social media services had confirmed on Twitter they were aware of issues and working to resolve them after thousands of people reported.
Users were eventually able to access Facebook and Instagram from late on Monday evening, while WhatsApp said its services were “back and running at 100%” as of 3.30am on Tuesday.
Facebook said a “faulty configuration change” on its routers was believed to be at the centre of the outage.
It said in a statement: “Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centres caused issues that interrupted this communication.
“This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt.
“We want to make clear at this time we believe the root cause of this outage was a faulty configuration change. We also have no evidence that user data was compromised as a result of this downtime.”
The platform added it was working to understand more about the outage in order to “make our infrastructure more resilient”.
In a statement posted to Twitter at 11.31pm on Monday, Facebook apologised for the outage and thanked its millions of users around the world “for bearing with us”.
It said: “To the huge community of people and businesses around the world who depend on us: we’re sorry.
“We’ve been working hard to restore access to our apps and services and are happy to report they are coming back online now. Thank you for bearing with us.”
Data on the web service monitoring platform DownDetector showed that almost 50,000 people had reported the outages on Facebook by just after 5pm.
Most complaints cited issues with the website (72%), while others were linked to issues with the server connection and the app.
Moe than 75,000 had complained about WhatsApp, with 43% reporting issues with the app itself, while 28% cited the server connection and 28% relating to sending messages.
More than 30,000 Instagram users also had similar complaints, with 51% relating to the app, 26% over the server connection and 23% citing the website.
Facebook’s share price plummeted 4.9% amid the outage, which also came the day after a whistleblower claimed in a US interview that the company prioritises its own interests over the public good.
The platforms were affected by outages in 2019 – in an incident which saw #FacebookDown, #instagramdown and #whatsappdown trending worldwide on Twitter.
A few months prior, users of Facebook and Instagram reported being unable to open pages or sections on the apps.
The issue also affected other platforms such as Twitter amid an increase in traffic on its website and app.
Twitter Support tweeted: “Sometimes more people than usual use Twitter. We prepare for these moments, but today things didn’t go exactly as planned.
“Some of you may have had an issue seeing replies and DMs as a result. This has been fixed. Sorry about that!”
It had earlier joked: “Hello literally everyone.”