Yep. This will cruise along longer than the parent's solution, but when it breaks, you'll be starting over all of the original services from scratch plus the management system you had built once to manage them.
But it only breaks when all systems fail together; if your router fails, you can rebuild it from the gitlab job. If the VM host fails, you have time to replace it because the rest of your network still functions. If your git host fails, same thing, but where did you put those backups?
My solution is Kubernetes. Everything's configured in YAML files. The solution to all those problems is... change fields in YAML files.
Of course, you need to figure out what you need to change and why, but you'll never not need to do this, if you're rolling your own infra. K8s allows you to roll a lot more of the contextual stuff into the system.
Do you find there to be a good amount of overhead in running your own Kubernetes cluster? I'd think initial setup would be a bit of work, and then keeping the cluster updated and patched would be a good amount of work as well.
Then you've just traded maintaining one system for maintaining another.
Just started this journey myself and while there’s tons to learn, getting something up and running using k3os and flux cd takes no time at all and gets you a working cluster synced to your repo. K3s is pretty light, I know some people running it on pis.
If you use hosted Kubernetes (GKE, EKS, etc) then you don't need to deal with any of that, which is nice. You get the flexibility of Kubernetes without needing to care about any of the underlying infra
Once you learn it it's pretty straightforward. K8s has a very simple underlying architecture. It's intimidating at first, but yields to study and care.
I have also been using Kubernetes for this for years now. My favorite property is that it will run forever, no matter what happens.
The annoying part is that when I do want to do updates (i.e. updating cert-manager from 0.1.x to 1.0.x, etc) it can be a pain. So I save these large updates for once a year or so.
The solution is keeping a local mirror of all images and artifacts, and version pinning for stability (along with a periodic revision of version numbers to the latest stable version).
This happened a few times to me over the years and then I was lucky enough to go on a packer/terraform course.
Now everything is scripted and stored in git. A Gitlab job rebuilds the VMs from scratch every two weeks to include the latest bugfixes and updates.
It was a lot of work at first but actually most of it was a learning experience.