Oh god they had to patch the Muppet, I had no idea anyone was running that in production anymore.
Once upon a time I also had to patch the Muppet, butt I did so in our application code, pre-save logic.
We had bootstrapped a multitenant online food ordering system into profitability, and used the Muppet for our menu data structures.
Things got expensive when our oldest customers a) added items to their menu and then b) reordered those items before their older items.
Under the hood, the Muppet uses sequential identifiers to determine which elements are ordered where in the tree.
The problem is that there is (or was at the time I dealt with this) one tree_id sequence for the whole table, so our oldest (and most invested) customers would drag a new item to the top of their menus and then...renumber all trees between the oldest tree and the newest tree being relocated, which caused massive writes across every intervening menu item. Sequentially. From Python. In individual statements.
I wrote a dash of code that would calculate the affected number of intervening trees, and if that number exceeded some arbitrary percentage of the table size, first relocated all of the tenants trees to the end of the tree sequence, and only then performed the tree-relocation of the new item into the preexisting menu, thereby absolutely minimizing the number of updates necessary.
Several days later, at the behest of my internal demons, with my favorite CTO ever cheering me on, driven by an irrational fear of table bloat, I shipped a patch that also found gaps in the tree_id sequence large enough to accommodate relocations to implement continuous defragmentation.
This code ran performantly and untouched until we shuttered the business after $megacorp acquired it and ran it into the ground.
Very interesting write up. Updating things is non-trivial as often you may depend on something that will eventually decide to deprecate a feature you actually used or some library/framework will decide to completely redo their API every five to ten years and then you suddenly have to start an upgrade in your project and try to keep this transparent to your users so things doesn't change for them or you need to figure some path to deal with backwards compatibility.
There are always an assorted amount of chores the more the project ages and has active users, these tasks often end up invisible as they don't get a featured spot in the release notes.
Choosing the right dependencies is a craft that needs to be learned. I find myself looking into the ppl. behind libraries more and more. I will always prefer something striped down from a known person in the space to the solution of a newcomer. Hoping between APIs every few years gets old fast.
In my mind I was specifically thinking of SDL doing the 2 to 3 transition, not really a newcomer. I also experienced this in the WebView transition from Qt4 to Qt5. I also still remember the python 2 to 3 transition in the beginning. There are definitely more cases of a very established library/framework breaking things.
Qt updates (especially when using something like PySide additionally) are always rough. I waited long enough with Python that the tooling for the switch got good. Large breaking changes luckily seem to happen less in the python ecosystem. There is a lot of movement in the dev-tools currently but CPython seems to really focus on gradual improvements - which I appreciate a lot.
Sorry for the confusion if you thought this was a rust project; here is the explanation I added to the text:
InvenTree is an Open Source (MIT), self-hosted inventory and part library management system written mainly in Python. It uses Django and DRF to expose a REST API that can be used from the web, iOS and Android apps, a Python library and IOT devices.
Once upon a time I also had to patch the Muppet, butt I did so in our application code, pre-save logic.
We had bootstrapped a multitenant online food ordering system into profitability, and used the Muppet for our menu data structures.
Things got expensive when our oldest customers a) added items to their menu and then b) reordered those items before their older items.
Under the hood, the Muppet uses sequential identifiers to determine which elements are ordered where in the tree.
The problem is that there is (or was at the time I dealt with this) one tree_id sequence for the whole table, so our oldest (and most invested) customers would drag a new item to the top of their menus and then...renumber all trees between the oldest tree and the newest tree being relocated, which caused massive writes across every intervening menu item. Sequentially. From Python. In individual statements.
I wrote a dash of code that would calculate the affected number of intervening trees, and if that number exceeded some arbitrary percentage of the table size, first relocated all of the tenants trees to the end of the tree sequence, and only then performed the tree-relocation of the new item into the preexisting menu, thereby absolutely minimizing the number of updates necessary.
Several days later, at the behest of my internal demons, with my favorite CTO ever cheering me on, driven by an irrational fear of table bloat, I shipped a patch that also found gaps in the tree_id sequence large enough to accommodate relocations to implement continuous defragmentation.
This code ran performantly and untouched until we shuttered the business after $megacorp acquired it and ran it into the ground.