Technologies:
Python (sync and async, Django and non-django), C, Java, Typescript, some C++, past in Rails.
AWS, Docker, Pulumi, Linux, a little GCP.
Résumé/CV:
10 years doing tech, past six in SRE roles (one FAANG, one startup). Lots of scalability and automation work in these roles, working with SWEs. GitHub is sheenobu.
A continuation is a function that, when called, jumps back to a specific part of the program that was frozen (meaning current memory, the call stack, the line number). So it can't really be stored. If your game state was stored in a db, you'd still have to load it memory to perform operations on it. Those operations, if using continuations, would then stick around in memory while waiting the continuation to be resumed. If it's a multi-player game, then you'd be in real trouble as your continuation could end up with stale data.
Game state has to be propagated from the server to the client so the player knows what is happening where-as continuations as used in this article are more about avoiding this propagation (the hidden field in the example is replaced by dispatch-table tag which acts as a session id / location in the dispatch-table for finding the continuation function)
Just to clarify, continuations don't have a snapshot of the whole heap, and they can be thought of as being a snapshot of just the interpreter state for a particular execution thread (roughly the call stack and source position). If heap objects are modified between creating a continuation and calling it, then the continuation will see the modified versions of all those objects. When it comes to garbage collection, continuations keep any objects referred to in their saved call stack alive. This is similar to any other closure, but instead of keeping things alive according to lexical scope, this is more of a dynamic scope thing (it's based on whatever functions happened to call this one).
Persisting a continuation seems like it's similarly as challenging as persisting an arbitrary closure. Like usual there are things you can't really persist, for example in your call stack you might have some unwind handlers for closing files -- how do you persist file handles?
It’s certainly possible to store continuations. You just store the program counter and a copy of the stack at the point where the continuation was captured. (There may be more efficient representations - I’m just talking conceptually here.)
If you google ‘serializable continuations’ you’ll find a reasonable amount of prior art (e.g. http://wiki.call-cc.org/eggref/5/s11n). I think there has never been a good solution to versioning then against changes to the code.
If someone builds a container that is designed like this:
FROM debian
RUN do-x
RUN install python2
and then someone changes do-x, if I understand, the layers below it get invalidated and all of a sudden install python2 fails. This is very bad design but very easy to replicate.
Okay, but surely my poorly written dockerfile that grabs the wrong images and runs broken code is not Debian's or python's problem?
Older versions of debian that still support python2 will theoritically be around forever, and any codebase that absolutely needs them should always work.
(though I contend that, given that we've had 15 YEARS of warning that this was coming, such instances should be vanishingly rare and not under active development)
> … "given that we've had 15 YEARS of warning that this was coming" …
This right here is the part that I'm still having troubles wrapping my brain around. People still stressing about Python 2 "going away" (it's gone folks; accept it) despite the fact that there's been well beyond a decade of advance warning, and Python 2 having been officially EOL ages ago now.
Spiritually it was the same concept though... if you can't get a driver for your Linux OS, use the driver from another OS. This just seems to take that to be extreme -- don't just use the driver from another OS, actually use the driver FROM the other OS!
I think I know specifically what you are talking about. The actual files an engineer could upload to populate their folder was not multi-region for a long time. The servers were, because they were stateless and that was easy to multi-region, but the actual data wasn't until we replaced the storage service.
I think the storage was replicated by 2013? Definitely by 2014. It didn't have automated failover, but failover could be done, and was done during the relevant drills for some time.
I think it only stopped when the storage services got to the "deprecated, and we're not bothering to do a failover because dependent teams who care should just use something else, because this one is being shut down any year now". (I don't agree with that decision, obviously ;) but I do have sympathy for the team stuck running a condemned service. Sigh.)
After stuff was migrated to the new storage service (probably somewhere in the 2017-2019 range but I have no idea when), I have no idea how DR/failover worked.
Thank you for the sympathy. If we are talking about the same product then it was most likely backed by 3 different storage services over its lifespan, 2013/2014 was a third party product that had some replication/fail-over baked in, 2016-2019 on my team with no failover plans due to "deprecated, dont bother putting anything important here", then 2019 onward with "fully replicated and automatic failover capable and also less cost-per-GB to replicate but less flexible for the existing use cases".
I have a new laptop running Arch, and Bluetooth headphones which nominally can act as a headset as well, but I actively disabled that functionality under Windows, because if something tried to use the atrocious-quality headset microphone, the high-quality headphones output would silently (literally!) not work, and only the low-quality headset audio output would work.
The headset modes haven’t appeared under Linux at all, which happens to suit me just fine, so I haven’t investigated the lack. A couple of times I’ve had issues with connecting at the BlueZ level, but putting the laptop to sleep and waking it up again has resolved it. (Just restarting bluetoothd doesn’t help.)
I was running PulseAudio for a few weeks, then I switched to PipeWire. I no longer get the Bluetooth indicator on it in waybar, but other than that the switch was fairly uneventful. One point is an improvement: it now seems to remember which devices I like to use, so that when I plug in my Yeti microphone it doesn’t switch audio output to it (it has a 3.5mm monitor port and can feed audio from the computer through that too), which it had always done under PulseAudio and I hadn’t yet gone to the trouble of figuring out how to stop it from doing that. One point in the Bluetooth device handling is potentially a slight regression: when silence should be being fed through the connection, I observe extremely quiet whine which I didn’t under PulseAudio. Not sure what’s at fault there.
My prediction is that once the capability reaches contact lenses, it'll become mass adoption. The physicality of wearing a headset while walking or moving is difficult for lots of people, and tiring. Contact lenses weigh a lot less, and are less intrusive to others.
Of course, it doesn't come without pitfalls, but that's just my prediction.
I think glasses/goggles are less intrusive than contact lenses. Lenses are irritating and difficult to insert. It's also impossible to share contact lenses. I think VR is already pretty close with the Quest 2, to what it needs to be to get mainstream adoption. A little lighter, a little thinner, maybe 5-10 more years of iterative improvements and we'll be there.
I wore lenses for 4 years and couldn't stand it. So I had them blast laser beams into my eyes in an attempt to fix it, so that I never had to wear lenses again. And it worked!
My hindsight also shows me that I trashed the WWW as "yet another system like gopher and WAIS that probably won't amount to much", so I should not be trusted with predicting the future, or even possibly the present.
There is nix, the language/package manager which can be used standalone; nixpkgs, the ports or homebrew like repo of packages that can be built/run on many systems including macOS; and nixos, the Linux+Nix distro that puts it all together.
I thought of Nix while reading this thread and I'm wondering what makes it unique here? As a daily NixOS user I get that it is better but I don't know the specifics. the nixpkgs rpeo is superficially similar to homebrew (lots of people submitting packages, running on github, automation around commits).
Well, a very simple answer is that homebrew embeds nonconsensual spyware into the brew tool itself, and nix does not. For me, "doesn't exfiltrate my private data to Google in the default config" is an important security benefit of nix over homebrew.
The longer answer is about the inherent benefits of the nix way of doing things; it is a horse of a different color compared to all other package managers I've seen
or heard about. It is a different installation paradigm, and the nix documentation (and many blog
posts) do a better job of describing its main differences than I can here.
Deterministic builds as a first class feature is probably the shortest summary. Being able to reference an entire and exact hash tree of deps is hugely valuable.
Analytics are an invaluable resource for a volunteer-run project. To their credit, they issue a noticeable warning with the command to turn it off. It seems to me your issue is more about using Google Analytics - if there's a better alternative that is sustainable (read: free and doesn't require much effort to maintain) that should be suggested.
My issue is that the data is exfiltrated without consent. It literally does not matter where it goes if private data is transmitted without consent.
With consent, it's fine to send it anywhere they like.
Their analytics are also unauthenticated. I sort of want to make the first letter of each package in the top packages ranking spell out something funny.
Fine on the unauthenticated part. But about private data - it's literally anonymous and just counting the number of installs of a package and the number of build failures. All the data they collect (and the code that handles it) is public. They can't even isolate individual users because no individual data is collected.
It's not anonymous by any stretch, that's a false claim.
It includes a persistent unique identifier, generated on install, a sort of Homebrew supercookie for tracking you across months or years until you wipe your OS install.
It also includes the client IP address (because you can't make an HTTP connection without transmitting that). The fact that Homebrew doesn't see this information doesn't mean it's not transmitted (to Google). This leaks your city-level location.
These two things permit Google to assemble a rough location tracklog based on client IP geolocation (correlated via the unique install ID), along with which packages you installed. Google, as we know, spies for the US federal government.
You're missing the point here, though: even if it were totally anonymous (which, as I've pointed out, it's not): it's still unethical malware even in that case because it's private data transmitted without consent. The fact that you don't consider your usage data private is fine; others do and transmitting that from their systems without consent is abuse.
I mentioned that it's unauthenticated to point out that there's literally nothing stopping anyone on the whole internet from polluting the dataset with whatever bogus information they want. I wouldn't undertake this myself, but it's entirely in-bounds for an organization that feels entitled to co-opt your private computer to conduct surveillance on you without your consent. It's a public API, after all.
We’re looking into other analytics platforms. If you happen to know a good replacement with better privacy and similar features, we’d be happy to review PRs.
Please mind that knowing unique installs will remain important for us though. We have zero interest in tracking people but we do need meaningful install counts. Those numbers have been super helpful for making decisions, for example which packages are worth maintainers’ time and which aren’t.
I don't care at all about how valuable the stolen data you've obtained by violating people's consent is to you.
You're shipping unethical malware. Making the data more private, switching analytics platforms, doing IP scrubbing... it doesn't change the ethical issue at all. You're stealing data without the consent of the subject/generator of that data.
What you're doing is illegal in medicine, for example.
Remote: Preferred!
Willing to relocate: No
Technologies: Python (sync and async, Django and non-django), C, Java, Typescript, some C++, past in Rails. AWS, Docker, Pulumi, Linux, a little GCP.
Résumé/CV: 10 years doing tech, past six in SRE roles (one FAANG, one startup). Lots of scalability and automation work in these roles, working with SWEs. GitHub is sheenobu.
Email: sheena.artrip@gmail.com