You’re ignoring the OS doing other things as well before letting you draw to the screen.

This isn’t a video game where they’re direct drawing to a graphics context with exclusive use of the resources. They’re also setting up native widgets in the background, coordinating data downloads etc..

Unlike a video game where you control the rendering and scheduling to a greater degree, here they’re beholden to when the OS deems it ready to redraw or scheduling of background processes that may move their task to a lower priority.

Yes you can go fast if you don’t factor in the rest of the system, but they designed Lottie to play well within a retained mode UI setup, which has inherent overhead.

I do feel like video games definitely have it harder on this front in general, but tend to have loads of people focused on performance "at all costs", including generating a bunch of messy code to get something shippable.

I also think saying that a retained mode UI has inherent overhead is a bit odd. immediate mode UIs have to rebuild their entire draw graph on each frame! meanwhile retained mode UIs have a bunch of metadata letting them get nice optimization improvements. Translating an object means you might not even need to redraw it!

There is definitely a certain truth to the OS being able to just force you down to a lower priority, but I think we all know that iOS is pushing the top app to as high a priority as possible. Meanwhile the "inherent difficulties" with something like AirBnB (getting data over the wire) are definitely inherent but this is why we talk about placeholders and stuff like that yeah?

But if you're in the business of selling rentals it's hard to have so many engineers focused on the performance of the app (especially when you are in the act of being successful already)

With regards to retained mode overhead, I mean it across multiple forms not just the cost to redraw.

There’s the overhead of pushing updates to the UI layer from your app code, having it trigger an update, which in turn makes it check what other things have updated and what needs to redraw. UIKit is handling all that for you, but as a result you’re not getting full control of when you’re updating. You get pacing issues, where you can’t really guarantee you’re drawing at set intervals either which can lead to pacing stutters since you’re beholden to the apps event loop.

With an immediate mode UI in a game, yes there’s more cost to set up your draws, but you have more direct control of when everything will update because you’re handling the event loop more directly.

As for other priorities, I don’t mean just app priority. I mean intra-app priority. Lottie has to mix in with user code, the app it’s within might schedule a task with a higher QoS priority. It’s not just showing a fixed animation, it’s part of the app interaction model. So it’s easy to get into a situation where the rest of the app introduces stutters inadvertently.

Using CoreAnimation lets you sort of side step that discussion.

And yes I agree games have it harder over all because they have to handle everything. That wasn’t my point really. I think though that games have it easier when it comes to frame timing as a result of being able to control the entire event loop. So it’s easier for a game to reason about and guarantee smoothness of motion than it is for a framework that has to fit into arbitrary applications.

Lottie is an industry standard cross-platform animations pipeline, its not limited to "apps AirBnB is making".

Irrelevant, the apps it's being used for would run fine on a 25yr old machine at 60fps.

You shouldn't need multi-threading to render a few thousand objects and most UI apps of the type Lottie is being used for animate 100 or less at a time.

Note: It's good that Lottie runs faster for its users. My point is only that the entire stack is over engineered and the solutions being used are because the stack is bloated and inefficient

I agree with the general point you're making, but a 1997 computer would not be able to render even just a basic loading spinner at 1170x2532 at 60 FPS. Software rendering would be impossible even with an infinitely fast CPU as it would exceed the bandwidth limit of AGP 1.0, which itself only came out that year. The graphics hardware at the time simply did not support that resolution at any speed.

This is a common problem when people complain about graphics performance and compare to older hardware - we forget how much higher modern screen resolutions are. It's not a smoothly scaling problem either. As often with scaling, you hit problems at particular levels that require a totally new solution to the problem. CPU speeds stagnated whilst screen technologies didn't, hence, complexity.

> their solution, instead of getting rid of the 1000 cuts is to apply more band-aids

That's engineering at scale. It's impossible to remove more than a small fraction of those cuts within any reasonable time frame.

Yep, same deal with AsyncDisplayKit and the classic "iOS can't handle our scale". Too many cooks. Move along, this isn't the library you're looking for

This makes me wonder: are immediate-mode GUIs (like Dear ImGui) subject to stuttering and frame-drops under compute-heavy workloads?

Yes they are. They’re beholden to whatever your event loop is.

they’re being told when to draw (hence the immediate), so if you suddenly have computation on the same thread that’s blocking OR your thread gets deprioritized , you’ll get stutters.

Immediate mode UIs aren’t very suitable for mobile use though. They don’t have great consideration for battery efficiency, unless you introduce a retained backing of some kind. They’re also not great for accessibility, though that’s mostly just because they’re not native.

Do you have any proof of this or just a gut feeling?

Immediate mode guis do cache stuff, at least Dear ImGUI, one of the most popular Immediate mode guis caches. Further, at least in my mobile usage almost everything I do with my phone re-draws the entire screen. The #1 thing to do is scroll through content. There's very few apps I run where only some tiny thing is getting updated. Maybe my music player.

I don't think it's settled that retained more GUIs are a battery win.

Once you’ve introduced a state based caching and differentiation layer, you’re introducing retention and are entering into hybrid UI. If you don’t have sufficient caching layers to figure out when something has to update, you’re drawing too often killing battery.

Maybe there are fewer purely immediate mode UI libraries today, which muddies the discussion though.

On the note of the apps used, I would say the vast majority of apps I run only have a few elements updating at any given time. Most of them are based on scrolling for navigation but that’s a small part of what I do. Photo viewing and editing, viewing sites, replying to messages or mail, listening to music. Very little is a full screen update, and if it is, UIKit and SwiftUI are caching large amounts of the view objects to keep things snappy and only doing it when they receive input that requires it. Can immediate GUIs do that too? Of course, but again you enter the domain of retention.

> you’re drawing too often killing battery.

You say this with zero proof. Checking 100s of widgets to see if they overlap, updating their damage boxes, drawing the parts of the ones that got overlapped, computing clip bounds for each one, switching graphics contexts, to do all that adds up to "killing the battery"

Retained APIs are like using a binary tree where an Immediate mode GUI is like using a vector. CS principles say the tree should be faster as insert and delete are in O(1) but in reality, cache misses and similar things kill all the perf you supposedly gained by over engineering the solution.

The same is often true of retained mode GUIs vs immediate, especially with all the transparency effects in modern UIs. Computing the minimal amount of parts can requires a ton more CPU than just drawing.

As far as caching = retained, no. The difference between an Immediate Mode and a retained mode API is if you, the user, have to create and maintain a tree of retained API widgets. No one is going to implement an immediate mode API and try to have a text widget that word wraps and expect it to have to compute all the word wrapping every frame.

Except you’d not be doing all those things that you mentioned. Much like you wouldn’t in a game, you’d be constructing some kind of acceleration structure to only update what’s needed.

With an immediate mode GUI, you have to typically draw most of the hierarchy when even a single widget changes unless you maintain a state based cache. You’re inherently calculating construction of more unless you’re also doing the same optimizations that a retained mode system does.

The tree vs vector analogy isn’t correct either because neither mode prescribe a data structure. I know it was meant to be an analogy but I don’t think it really applies because it doesn’t reflect most code bases and frameworks.

Maybe our definitions of immediate vs retained don’t align.

As for metrics, I don’t have any on hand, and that perhaps makes the discussion moot, but we’ve done multiple UI implementations on my projects, and immediate UIs don’t compare favourably for energy use until and unless you introduce a retained backing.

Indeed that’s what interfaces like Flutter and SwiftUI do by exposing a React+immediate style API while working over a retained base.

I wish I could share our metrics, and I recognize there’s no reason for you to take me at my word on that, but we have done significant work in this area to compare systems for our projects. For reference , we often make use of multiple APIs across our different apps like Dear,Imgui and SwiftUI depending on their needs.