Where was Facebook’s HugOps last week? Exactly • The Register



Opinion As of this writing, it’s been exactly 100 hours since Facebook, Instagram, and WhatsApp came back from emptiness on the internet. They had been gone six hours – or seven billion dollars, if you measure your life by Zuck’s net worth, which we don’t recommend.

The time for hot shots is over. We are now in the brief sober analysis window before everyone forgets what happened. Next Thursday, by current standards.

There are three angles that matter. One is the purely technical side – how a giant company, built on the most resilient network technologies ever created, just disappeared. One is what it teaches us about the importance of Facebook in our daily lives. The latter is what he tells us on Facebook itself, where it will go next and if its open-pit exploitation of societal values ​​for profit will continue.

The technical side is quite simply put. Someone, most likely an unlucky sysadmin, tried to check the backbone for availability, but a malformed command shut down access to the company’s DNS servers instead.

He did this by canceling the routes to Facebook that the internet knew about. Security controls to prevent this from happening were broken, there was no Plan B, and as an added bonus most, if not all of Facebook’s internal tools, communications, virtual and physical access systems. were also closed. Hilarity ensued.

Did it matter?

Yes and no. There are many, many small businesses that rely on Facebook pages to talk to their customers. They had six hours of uncertainty.

WhatsApp has been widely adopted as a team channel by all kinds of people who have had to wait to plan, schedule, or dispatch work.

Families struggling with long-distance illnesses or crises have suffered.

You don’t painlessly turn off a multibillion-user service, and most of it has been felt by those least heard. But think about what would happen if Google, Microsoft, or Amazon suffered a total service outage like this. Even the partiel those we have seen created a universal concern that was absent here.

Instead of worry, there was a sigh of relief all around, and a wave of schadenfreude among the peers of this hapless administrator. It is unusual and remarkable.

Have mercy on the lonely and unloved Facebook administrator

Most sysadmins and other types of operations have enormous sympathy when an internal escape by one of the clans causes public connections. There but for the grace of God is the rule – and the #HugOps tag is deployed in fraternal and sororal support. This time, not so much. Normally everyone in the company recognizes that work is work, but the social network looks like a social death for network administrators.

This lack of respect amplifies the technical aspects of the failure. The raw components that Facebook is built from, the technologies and the people in the engine room, can do much better when allowed. You can create secure backdoors that bypass your main networks. You can create watchdogs that return to a known safe state when things stop happening.

You can’t predict everything, but you can assume systemic failures from whatever cause. Yet one single point of failure destroyed everything. Efficiency won out over redundancy. The worst case that never happens has happened. Too much security is never enough – until it is simply too much.

And these are management decisions.

When we look at how Facebook’s business model is built, we see the same thinking. Concerns of wrongdoing from the outside world should be ruled out, no matter how badly internal systems behave. Maximum efficiency, a resolute approach to collecting data and selling it – allegedly without nuance or backup – is the only real path.

Whistleblowers are the enemy. The media needs to be ignored, shunned or treated like idiots, and what do regulators or governments know anyway? Best practices are what Facebook says, not what everyone decides.

It is pride, characteristic of a company too powerful to care. State-owned telecommunications operators had it. IBM in the 1960s had it and Microsoft in the 1990s – neither is in any way cured. And Apple, Google, Amazon all have it.

But where Facebook is particularly vulnerable is that if it goes, it doesn’t matter much, even in the medium term. It’s a social network, and nothing else, and users can rebuild that social interaction on a different platform in a matter of weeks. Others are willing to shoulder the advertising and analytics expenses.

Unlike Microsoft, Amazon and Google, unlike telecom operators and IBM, Facebook does not provide essential services to businesses or the state. Quite the contrary – he threatens, and he has the money, but he has no leverage. A loss of trust at the top, or a sufficiently powerful legal or regulatory challenge, and it could disappear as fast as its BGP routes, an irrelevant Yahoo! for the 2020s.

Facebook is the single point of failure. He has no friends he can’t buy, just enemies he can’t pay. Six hours on a planet without Facebook seemed like six thousand hours too little, and now we know it. Facebook? ®



Comments are closed.