Tag Archives: Technology

Microsoft Outage Poisons Guests

Microsoft’s outage has now actually poisoned people, if this story is to be believed:

https://www1.wdr.de/nrw/rheinland/rhein-erft-kreis/technischer-defekt-chlor-phantasialand-100.html

It’s in German so you’ll need to run it through Google Translate or similar. This link might work instead:

In short, the Microsoft outage took out Outlook and the park’s maintenance systems, including the Pool maintenance systems, which sent alerts as emails, and emails were down due to the outage.

As a consequence, the staff were not alerted when too much chlorine was pumped into the pool.

15 guests were affected, with 5 (later corrected to 4) taken to hospital.

This is probably one of the few times a cloud outage has hospitalised people. Wonder how long before it kills someone. But then again, if you are putting a life-saving system in the hands of a cloud provider, you are asking for trouble.

Someone Turned Europe’s Supermarket Freezers Up to 50°C

Overnight into Monday 2 November, someone went into the refrigeration controls of supermarkets across Europe and turned the cold off. Stores in Belgium, Sweden, Estonia and several other countries were hit, with individual chains reporting anything from a single site to several dozen. Some pulled chilled and frozen stock from shelves; others threw it away.

This morning on the way into work, the Sainsbury’s in the station near where I work had its doors closed and a sign saying “due to technical issues we will be open at 07:30”. I wonder whether it is related to this.

Source: The Hacker News — New Wave of Refrigeration Cyberattacks Hits Supermarkets Across Europe.

Podcast Episode: Running Plans And Cloud Outages

Pip: Blender Fox has been busy — lacing up running shoes, watching cloud infrastructure catch fire, and apparently doing both in the same week.

Mara: Today we're covering two distinct territories: a return to a running habit and what those early sessions actually look like, and a Google Cloud outage that took down a GKE cluster and a MongoDB node at the same time. Let's start with the running.

Returning to the Road

Pip: The central tension here is a familiar one — you used to run, then you stopped, and now the apps you relied on have either died or drifted into something unrecognizable. So how do you actually structure a return?

Mara: The post frames it plainly: "It's been a long time, but I'm going to try to get back into running. Most of my old apps have died or no longer serve the original purpose they had."

Pip: That's the infrastructure problem of fitness habits — the scaffolding you built last time doesn't survive a long gap. You come back and have to rebuild from scratch, tooling included.

Mara: The practical answer here is Runnah, a new app, approached with deliberate caution — trial first, then monthly, then annual only if the habit proves it'll stick. The app also surfaces strength workouts, which fits the situation: a barbell and dumbbell set already on hand.

Pip: Choosing a beginner plan when you're not a beginner is one of the smarter calls in fitness.

Mara: The post flags that directly — not new to running, but far enough out that a slow re-entry makes more sense than picking up where things left off. The first session reflects that: five minutes walking warm-up, three sets of three minutes running at conversational pace with two minutes walking between, then a five-minute cooldown.

Pip: The second run, covered in the follow-up Training post, adjusts the intervals — ninety seconds at conversational pace, ninety seconds walking, then a four-minute conversational pace block, two minutes walking, same warm-up and cooldown bookends.

Mara: And it landed well. The post puts it simply: "Today's run was nice. Comfortable, with some harder uphills in the middle." The structure is doing what it's supposed to — accumulating time on feet without blowing up the effort level.

Pip: Two runs in, a new app on trial, a plan that's working. The question now is just consistency.

Mara: Which is exactly where these plans either hold or don't. Speaking of things that don't hold — let's talk cloud outages.

When the Cloud Has an Off Day

Pip: Sometimes your infrastructure just decides it's done for the afternoon, and there's genuinely nothing you can do about it.

Mara: The post describes a GCP regional failure in London that took down part of a GKE cluster and degraded live services. A MongoDB Atlas node — running on GCP but outside the main account — also went down. Atlas's own message made the situation clear: "The Cloud Provider you've chosen is out of capacity for the selected database deployment configuration. We will continue to try and deploy on the chosen Cloud Provider."

Pip: So the recommended fix is to pick a different provider or region — except you can't action that mid-incident when the provider is the problem.

Mara: The saving detail is that the affected Mongo node was an analytics node, not serving live traffic. Most services were back up by the time the post was written, though GCP's own status page didn't flag any issues until around two hours after problems first appeared.


Pip: A slow return to running and a cloud region that needed a lie-down — not a bad week's worth of material.

Mara: The running story is still early. Next sessions will show whether the plan holds.

Domain Transfer

My WordPress domain is coming up for renewal soon. I might migrate the domain off WordPress since I’m finding it very restrictive being unable to manage my DNS entries via the WordPress UI. This may cause the blog to be unavailable for a while, but I’ll drop a note here before I start.

Fedora 39

I decided to try upgrading my Fedora to Fedora 39 (from 38)

I ran into a few issues already.

  1. I used the option from https://docs.fedoraproject.org/en-US/quick-docs/upgrading-fedora-offline/#sect-performing-system-upgrade to download and upgrade the system using the “on boot” update. This then caused the initrd issue I’ve encountered before (https://blenderfox.com/2023/05/02/fedora-5/), I fixed that by booting up the previous version, then finding I couldn’t log in unless I picked “Gnome on XOrg”.
  2. The breakage of Wayland login seemed to be related to Problems with wayland after updating to fedora 39 – Fedora Discussion. And interestingly was fixed in the same way: removing ~/.config/dconf/user, logging out and then back in via Wayland. It completely reset my Gnome state, so all my custom docked shortcuts were gone and it offered to run me through the Gnome tutorial again
  3. Running my ansible playbook to configure my machine again, seemed to run for the most part, but failed at installing a pip module — but that looks like a module build failure, not the playbook.

And now I discovered that there’s no F39 of VirtualBox yet, so I guess I’m rolling back for now….

Terra-fork

Okay, so I swiped the headline from the video, but it’s pretty much a given.

Further to my previous blog entry about Hashicorp’s decision to change Terraform’s license, I mentioned how the OpenTF Initiative put a choice to Hashicorp: change the license back or we will fork Terraform.

Unsurprisingly, Hashicorp didn’t comply, so now Terraform has (or will be) forked as OpenTF at https://github.com/opentffoundation

This story has been covered in several places including The Register and several videos on YouTube including a nice short summary by Fireship

If the fork goes ahead, I’m curious as to whether Hashicorp sees it as a competition and goes after the foundation, but if it does, what little reputation Hashicorp has left in the OSS community will be nigh on destroyed.

Meanwhile, I’m going to continue playing with Pulumi in case the OpenTF doesn’t work, gets blocked or if we need another alternative.

Google Domains and Terraform

Two major updates recently

Firstly, as suspected, I finally got the notification saying that Google Domains’ registrations would be acquired by SquareSpace so all my registrations would be transferred over them should I not do anything. Obviously, I didn’t want that, so I transferred them over to AWS (Route 53) so now my domains are registered with AWS, but are DNS managed by Cloud DNS. I had some weirdness when trying to migrate all my domains in bulk, with respect to the auth code not being accepted on one of the domains when doing the bulk migration, but it was accepted when I did the migration on that one domain alone, so… go figure.

Next update is Terraform. In case you didn’t know, Hashicorp has changed the Terraform license and essentially made it no longer open source. This behaviour is similar to what Red Hat did with its RHEL offering and the backlash is just as bad.

Immediately I knew someone would fork it, and already, there’s the OpenTF Initiative and this is the key part:

Our request to HashiCorp: switch Terraform back to an open source license.

We ask HashiCorp to do the right thing by the community: instead of going forward with the BUSL license change, switch Terraform back to a truly open source license, and commit to keeping it that way forever going forward. That way, instead of fracturing the community, we end up with a single, impartial, reliable home for Terraform where the whole community can unite to keep building this amazing ecosystem.

Our fallback plan: fork Terraform into a foundation.

If HashiCorp is unwilling to switch Terraform back to an open source license, we propose to fork the legacy MPL-licensed Terraform and maintain the fork in the foundation. This is similar to how Linux and Kubernetes are managed by foundations (the Linux Foundation and the Cloud Native Computing Foundation, respectively), which are run by multiple companies, ensuring the tool stays truly open source and neutral, and not at the whim of any one company.

OpenTF Initiative (https://opentf.org/)

Essentially, make Terraform open source again, or a fork from the MPL version will be made and maintained separately from Hashicorp’s version. This will essentially lead to two, potentially diverging versions of Terraform, one BUSL and one MPL licensed

I’m already looking at alternatives and the two currently that I’m looking at are Ansible and Pulumi

Ansible I’ve had experience in , but there’s two main issues with it:

  • It’s underused compared to Terraform and the providers are woefully undersupported and undermaintained
  • It’s Red Hat

Pulumi I’ve heard lots of good things about, but its a new technology, and I don’t know if it is can “Import” existing infrastructure.

Guess a “spike” is worth doing for it.

Updates

It’s been quite a long time since I did any updates on this blog so a couple of updates are in order

House

I’ve now been living in the new place for just over a year. Generally everything is good, we’ve started putting in a lawn and currently letting it get its roots in before we try to cut it

Twitter

Oh, boy, what an absolute train wreck. I’ve tolerated Elon’s presence at Twitter because most of the stuff he did wasn’t far off what Jack was doing prior. But cutting off all third party clients, forcing everyone onto the new TweetDeck (which likely will be paywalled too) has literally driven users away. Twitter has been losing users and revenue constantly, and it is totally unsurprising.

I’ve disabled my two TweetDeck profiles (a professional one, and a casual one) from Ferdium. But I doubt I will be going back any time soon.

Red Hat

Another train wreck of a situation. Red Hat’s decision to first kill CentOS’s stability, then for RHEL sources to only be accessible behind a subscription has pissed of a lot of users, even if it’s not strictly speaking against license.

The only one left in its sights will be Fedora so that leads me onto the next update

Manjaro & Archlinux

I’ve been tinkering with Manjaro more and more lately, with its rolling release schedule meaning I never need to upgrade from a major version to another major version.

It’s downside I’m finding is that some packages, especially those in the AUR are essentially “compile from source” packages which does the build on your machine during the install. This can take a varying amount of time depending on the code. With my RSS reader of choice: QuiteRSS, this build takes a mind numbing 2.5 hours to do, even on a high spec machine.

That’s where I found out about setting up your own Arch repo. I’ve been tinkering with that, setting it up on GCP and fronted by a CDN. This works pretty well, but I still need to find how I can do scheduled builds to keep that up to date, but it looks like I’ll be switching to Manjaro at some point in the near future. Sound works fine, using lyncolnmd‘s work.

Another downside with Manjaro, however is that its btrfs filesystem, my home directory backup, and CloneZilla don’t seem to want to work well together

WordPress & Domains

One final update, I will likely be transferring out the blenderfox.com domain out of WordPress, while I had this registered as part of the blog, I’m finding it much harder to maintain this domain using WordPress’s very limited DNS management tools. I will likely transfer it out to Google Domains, even though there’s talk of them shuttering that service. Secondary service would be AWS.

Citymapper

I paid for the premium version of Citymapper primarily because I need to use their “speak directions” option while on public transport. Something that used to be free, and then they put it behind a paywall.

Now they only went and made it FREAKING FREE again, after I’d gone premium and paid for the year >_<

https://citymapper.com/news/2589/citymapper-club-features-are-now-available-to-all

So I’ve terminated the premium subscription, but I’m still premium until the end of this annual cycle, which ends in November