Not often I post on problems at Google, but this is actually an interesting situation.

https://arstechnica.com/?p=1518703

Google had an outage the other week, and it knocked out several websites GitLab, Shopify and impacted others. Gsuite, Gmail, YouTube were affected, but not down.

There are some interesting lines in this article:

for an entire afternoon and into the night, the Internet was stuck in a crippling ouroboros: Google couldn’t fix its cloud, because Google’s cloud was broken.

Google says its engineers were aware of the problem within two minutes. And yet! “Debugging the problem was significantly hampered by failure of tools competing over use of the now-congested network,”

In short, Google Cloud broke due to congestion, Google couldn’t fix the problem because their tools required using the network that was now congested