You have a function cleverly designed so that being zero is optimal. Closer to zero the better. It has 1000 dials to control it bit otherwise input and output.
So like a AWS Lambda with 1000 env vars!
Some clever math gal designed it so if you do this gradient descent thing it learns! But let's not worry about that for now. We just want to understand gradient descent.
So you have an input you like. And a desired output and thia function that makes an actual output and a way to turn that into a score of closeness. Closer to Zero better.
So you put the input, env vats, output and you get say 0.3
Not bad. But then you decide to wiggle an env var just a bit to see if it makes it better. 0.31 doh! Ok the other way. 0.29 yay! Ok so leave it there and do the next one and so on.
Now repeat with the next input and output pair.
And again with another.
Then do the whole set again!
You will find the average amount you are wrong by gets better!
This is sort of gradient descent.
One extra trick. Using maths and calculus you can figure out how to adjust the env vars so you dont need to guess and the amount you adjust them will be more optimal.
Calculus is about the rate things change, and if you say do A + B then a change in A becomes the same change in A + B but you can also do this in reverse! This let's you calculate not guess those changes needed to the env vars.
Imagine you're in a hilly landscape and want to go down as far as possible, but you can only see a small area around you, so you can't just directly go to the lowest point, because you don't know in which direction it is. Gradient descent is based on the idea of looking at which direction your local area is sloping upward (the gradient) and jumping in the opposite direction (i.e. down) a distance proportional to the strength of the slope.
This works well when the slope near the lowest point gets flatter and flatter, so that gradient descent makes smaller and smaller jumps, until you reach the bottom and stop. But if you end up on a very steep wall, you would make a very large jump, maybe so large that you overshoot the target and end up on an even steeper wall, make an even larger jump in the opposite direction and so on, getting farther and farther from the goal.
So one idea is to make sure that your jumps are always small enough that even the steepest wall you could possibly encounter won't throw you into a vicious cycle of increasing step sizes. For example, if you pour water on the ground, the water molecules make truly tiny jumps flowing down to the bottom, and in the article they call this path the gradient flow.
But what they show is that gradient descent typically splits off from this smooth gradient flow and instead gets into an area where the jumps get bigger and bigger for a while, but then the cycle is broken and they get smaller again. That is surprising! Even though it seems like the jumps can only keep getting farther, somehow they must've gotten close enough to the goal to calm down again.
So what the authors did is to remove the direction in which the jumps get bigger and look at what happens in the middle. You can imagine this as a valley with steep walls and a river smoothly flowing down at the bottom, and the jumps go back and forth across this river.
They call this river the central flow and show that it doesn't only flow along the direction of the gradient, but also a little bit in a direction where the steepness decreases. So when the jumps cross the river, they're also moving downriver a little, until they get to a point where the valley isn't so steep anymore and the jumps get smaller again.
NotebookLM is a success in spite of the interface not because of it.
Pretty awful UX to be honest.
Credit goes to the backend engineers who made this happen.
Took me ages to figure out you can copy paste simple text. Editing any text isn't great either.
Why isn't there a dedicated markdown notes section with folder and files? Most of us won't have ready to use PDFs but almost all of know how to use a text editor and paste text into it. Your Notes app UI is without proper support for editing and maintaining notes.
Are you a middle aged+ developer? You read the DDD/CQRS/event driven book or viewed the conference. My dear friend, you have fallen victim to developer porn. It's very addictive. You end up playing "the architecture game" rather than building anything useful.
I have yet to come across ONE good large scale project that makes use of these and isn't a big ball of mud.
I wish I had the time to do any of this. I could probably do it on a weekend but maintaining it, upgrading it to keep up with new releases would be something I wouldn't have time for.
I end up just paying a cloud provider and forget about it.
Anyone else on the same boat? What has been your approach?
Honestly, I self-host about a dozen services and upgrades take me less than a minute per month usually.
I simply have one folder per service, each folder contains a docker-compose stack and a storage directory. Updating is simply a matter of running `docker compose pull` and `docker compose up -d`. Nothing more.
Breaking updates requiring to tweak the config are very uncommon, and even when they happen it's only a few minutes of checking the updated config and applying it.
IMO this is the simplest way to self-host. No VM, no complex software install, nothing more than a simple Docker Compose setup that's fully automated.
That sounds similar to my setup, but each folder is a btrfs subvolume and my update script takes a snapshot before updating. I keep the Docker compose file together with the volumes in that subvolume.
If something breaks I can decide to figure out why, or revert.
With previous setups, I was certainly guilt of not upgrading and doing the maintenance needed. That's one reason why I like using NixOS and ZFS. Both provide really easy rollback options. So all I need to do is run an update and rebuild. If things work, no more for me to do. If things don't, I can try debugging or just revert back to the previous release till I have time to
But also I think using a cloud provider is fine if you're happy with the experience. It is a time sink to get things setup and it's not zero maintenance time. It's reasonable to weight those costs
It's usually not a single weekend. If you're like me, it starts out with thinking it'd be nice to install Plex on an old gaming PC. A year later, it has organically grown into a rube goldberg machine of proxmox and home automation. Which I guess just reinforces your point.
Joking aside, a minimal setup just using docker compose is pretty manageable. Self hosting many projects is as easy as 'docker compose up -d', and upgrades are straightforward as others have pointed out.
After working with NATS, I wouldn't want to touch Kafka even with a long stick. Its just too complex and a memory hog for no good reason. It doesn't have all the features that NATS supports as well.
I skipped using aider, but I heard good things. I needed to work with large, complex repos, not vibe codebases. And agents require always top-notch models that are expensive and can't run locally well. So when Codex came out, it skipped to that.
But mct leverages the weak models well, do things not possible otherwise. And it does even better with stronger models. Rewards stronger models, but doesn't punish smaller models.
So basically, you can use save money and do more using mct + codex. But I hear aider is terminal tool so maybe try and mct + aider?
Not racist but offensive indeed. It's like relating school shootings to white people. A white child sitting in Norway might not relate and may find it offensive and insulting.
India is a country, not a color of person. It's more like relating school shootings to the US, where an American child sitting in the US would definitely relate and find it accurate.
