I can provide some details regarding the times things did break that I mentioned in the article.
* In September 2014, X broke, and I created an `/etc/X11/Xwrapper.config` file with the lines `allowed_users = anybody` and `needs_root_rights = yes` to get it to work again. I don't remember and don't have notes on why that helped. It sure does sound like a pretty terrible hack. I don't have that Xwrapper.config file anymore, and I also don't know when I deleted it.
* In June 2017, audio stopped working, but all I had to do was add my user to the `audio` group.
* In May 2018, X broke a second time. This time I downgraded the `xorg-server-common` and `xorg-server` packages. A few weeks later, I ran another system upgrade, and this one went fine.
These weren't the only problems, but they were the most disruptive. Generally, things like TrackPoint driver updates changing how the cursor responds or Firefox changing its UI have been far more annoying than Arch Linux issues :)
I had a similar life with arch. A handful of boot blocking issues, let's say 5. 4 out of them were solved after joining #arch on now-dead freenode and realizing this was explained on arch main page. 1 of them was a deeper borkage that arch team didn't catch early and required a bit of surgery. The problem was gone in 5 minutes.
That's not the place to explain it. If they're going to require user intervention on updates, the place to do it is during the pacman -Syu itself. Better yet have a shell script that'll fix it automatically and give the end user the ability to read the source before execution.
The worst incident I can remember from several years of running an Arch VPS is when they had relocated some /lib or /usr/lib files and it required intervention, and I did a pacman -Syu and rebooted without reading the news. Just a mild PITA in OOB console.
I definitely had some rough edges with the pulseaudio, and then pipewire, upgrades, and a few cases where almost everything broke because in my infinite genius I had compiled my own (insert dependency) for a bleeding edge feature, forgot to revert when it made it to mainline, and then later down the road a major version bump meant some `.so` was missing, and I had to USB liveboot to fix it.
I've also been on Arch for over a decade, and it's almost never been broken, even when I was playing with some seriously bleeding edge components. Almost always, it's been surprisingly straightforward to un-screw the few screw-ups I've made.
> Almost always, it's been surprisingly straightforward to un-screw the few screw-ups I've made.
That's been my experience as well. With Arch, everything is exposed. There are usually no wrappers or layers of abstraction or weird modifications added to the upstream components it ships. That makes problems so much more straightforward to troubleshoot and fix.
Exactly. And having gone through the install process to the point of a usable desktop, I learned a ton about what I'd need later to fix the issues myself, quickly. And then when everything just worked, much better than Ubuntu, I installed Arch on a few other machines and got even more practice.
The learning curve may have been steep but it definitely paid off in my understanding of how to maintain and fix issues going forward. And just my understanding of Linux overall.
I yearn for my arch days where 'ls /etc' only yielded things I knew about.
These days I am stuck with WSL, and that sadly does not work with Arch. As far as I can tell the Arch community does not want to support WSL because of philosophical disagreement.
That is not the kind of packaging that inspires confidence. And the effort it takes to verify it isn't backdoored is too much. Definitely, my IT department is not going to appreciate me installing this. In this case, I actually think our IT department would be right.
the pipewire thing has been a debacle on gentoo as well, with it being a coin toss for at least this year whether a full software update will break audio and video or not. And i didn't write a note anywhere except on the gentoo matrix channel. And if you've ever tried to search a busy matrix channel on a single-instance matrix server...
If memory serves, deleting some file from /etc/ fixes the issue.
The glibc upgrade which was painful (and essentially required recompiling everything) was much further back than 10 years. I think I was running LFS at the time, but recall it was painful for all distros. I don't think there was a glibc upgrade that was disruptive since then. There was the introduction of multi arch on Debian some years back which caused a bit of disruption (I was running Debian unstable at the time IIRC), but pretty much everything else has been very minor since then. I've been running arch for 2 years or so now, before I was running tumbleweed. I have to say that rolling releases have been much less eventful in general than release based distros (I administer my partners Ubuntu laptop and lts upgrades are a bit more disruptive).
The current glibc version broke Easy AntiCheat support for Proton games, but that's the only break of note in recent memory, and it only affects people playing multiplayer games on Linux, which is a minority (multiplayer gamers) of a minority (Linux gamers) of a minority (Linux users).
glibc updates have recently broken lots of Electron software (and probably other stuff using similar sandboxing), by using a new syscall (clone3? or something) to implement some library methods.
Pretty much every glibc update breaks something, honestly.
That breakage is because of the dumpster-fire that is seccomp. Your seccomp policy (in this case, the one that comes with Electron) whitelists syscalls, but which syscalls glibc uses to implement things is considered an implementation detail, not part of the contract. So seccomp was designed in a way that makes it broken-by-design with the most popular libc.
In Arch, the glibc package upgrade associated with the `/lib` + `/usr/lib` merge in 2012ish was painful. I assume that's what the parent post was referring to. I assume you're referring to the libc5→libc6 upgrade?
For many users it was a non-event; but if you missed that news post, and didn't pass `--ignore glibc` to your `-Syu`, then your system broke. And a sizable minority of users missed the news post. (Shamefully, I was in that minority.)
Yes I think you're correct libc5 to libc6 is the upgrade I was talking about. I just had a look that's more than 20 years ago. Funny how people still talk about "painful libc upgrades".
I suspect you were thinking of an Arch Linux-specific glibc upgrade related to Arch's `/lib`+`/usr/lib/` merge in 2012ish, not a painful glibc-itself upgrade that other distros would have noticed?
glibc and the file system update were both horrible updates in Arch.
The systemd transition was annoying as I liked Arch’s unit system but it was just an annoyance rather than breaking change.
Recently I ran into an issue were I had to revert to a LTS kernel because the main kernel hangs during boot each time (spent hours debugging and haven’t found the culprit. But the LTS kernel is working fine so I’m going to stick with that).
Also, Nvidia gpu drivers are the worst (I was on Manjaro back then) to seeif I could get rid of Windows for gaming purposes. I used Linux for games for about 6 months and had to quit and get back to Windows.
Proton is a game-changer, but Nvidia drivers remain the most unstable thing on Arch. Find a version that works well with your card, and avoid upgrading it, if at all possible. It's performant, but I have games that crash once every few hours, but only specifically on certain machines.
This is true everywhere. I have some very expensive Lambda Labs GPU blades, and even their Lambda-stacked Ubuntu upgrades break CUDA stuff occasionally. I think Nvidia's driver ecosystem is held together with chewing gum and duct tape.
I use NVidia on my single Windows gaming system, but every Linux desktop system around here has AMD. I actually end up playing just about any game that plays on Linux on one of my Linux machines, and only boot the Windows system when wanting to play an exclusive title or occasionally just to do OS updates to be ready to play later. I've been using almost exclusively ATI/AMD GPUs since the Mach32 days, but for Windows gaming systems sometimes NVidia has the performance crown for months or years at a time with reasonable stability.
My fastest laptop is an oddball. It uses an AMD mobile processor with integrated graphics and an NVidia discrete GPU. Windows 10 handles that fine. Several Linux distros didn't install correctly at the time I bought it, but Ubuntu did. I'd have been okay with it installing to only support the integrated GPU, but it supports both and has a menu option for every app to launch it on the dedicated GPU. So far the only problem I've had with that whole laptop is a Windows update breaking the bootloader and making Linux temporarily inaccessible.
Been on Manjaro+NVidia for a few years now. Gaming works very well, especially for recent games (2017+). Most "Windows only" games play perfectly. Older games are hit or miss, but misses are pretty rare.
My first significant Arch install was also pretty long-lived (not 10y but def >5)
Did you go straight to full-on systemd when you installed it? Arch was transitioning to systemd by default around the time of your initial installation (default since Oct 2012 so you would have just missed it if you went with defaults).
Mine was a few years of cruft-accumulation in already and my init system understanding was not super deep at this point so this (together with the glib thing around the same time) is the most disruptive upgrade breakage I recall. If it doesn't show up in your top 5 I'm guessing you dodged it?
I had a couple of storage/media servers on Arch that I set up in 2008. In 2012 when systemd came, I experienced lots of pain. I eventually got things working but it took me a long time not to hate systemd. They both failed within a few months of each other in 2018, putting them right around 10 years of Arch as well.
I had similar, but more serious, X breakage with Debian testing. I used the apt feature to clean up orphaned packages, but the dependencies weren't listed quite correctly in the packages. I ended up spending an hour or two chasing dependencies manually to reinstall the missing packages. So Arch is definitely not alone in things sometimes breaking.
* In September 2014, X broke, and I created an `/etc/X11/Xwrapper.config` file with the lines `allowed_users = anybody` and `needs_root_rights = yes` to get it to work again. I don't remember and don't have notes on why that helped. It sure does sound like a pretty terrible hack. I don't have that Xwrapper.config file anymore, and I also don't know when I deleted it.
* In June 2017, audio stopped working, but all I had to do was add my user to the `audio` group.
* In May 2018, X broke a second time. This time I downgraded the `xorg-server-common` and `xorg-server` packages. A few weeks later, I ran another system upgrade, and this one went fine.
These weren't the only problems, but they were the most disruptive. Generally, things like TrackPoint driver updates changing how the cursor responds or Firefox changing its UI have been far more annoying than Arch Linux issues :)