It's kinda awesome that after decades of software and hardware advancements to prevent computers from arbitrarily executing data as instructions, we've decided to let agents arbitrarily execute data as instructions.
Or find it surprising that probabilistic tool based on generating things can do things when you give it rights to do things... And that you can not effectively program it to not do something....
You gave it capability to delete emails. Why did you expect it not to do that at least some of the time? And with enough user some of the time will most likely happen...
This reminds of the conversation the other day about the deleted production database at railway. "this person obviously didn't follow best practice of being hyper distrusting of LLM agents", and the response "yeah but every company is marketing it as safe. someone is gonna fall for it".
(Well-regulated) free markets are sort of built on the principle of educated consumerism. Your choice matters; its not up to the government to make illegal every non-optimal product. However, we do expect some minimum level of safety.
What does that mean for llms? Their nondeterminism does seem to incline them toward a legal safety requirement. Can you buy a fire extinguisher that 1/1000 times burns your house down? Or can your car brakes instead increase acceleration in rare cases?
Im using llms much more than i used to, but i still cant shake the fundamental stochastic nature of the technology.
Wherever I'm going, I'll be there to apply the formula. I'll keep the secret intact.
It's simple arithmetic.
It's a story problem.
If a new car built by my company leaves Chicago traveling west at 60 miles per hour, and the rear differential locks up, and the car crashes and burns with everyone trapped inside, does my company initiate a recall?
You take the population of vehicles in the field (A) and multiple it by the probable rate of failure (B), then multiply the result by the average cost of an out-of-court settlement (C).
A times B times C equals X. This is what it will cost if we don't initiate a recall.
If X is greater than the cost of a recall, we recall the cars and no one gets hurt.
If X is less than the cost of a recall, then we don't recall.
But intelligent beings are fundamentally fallible? That's kind of the nature of doing leaps of reasoning: sometimes those leaps are amazing, sometimes they're wrong. It's what's advertised.
We're in the same era where lots of peoples' installation guides for the software they want people to use is essentially boiled down to "sudo curl | bash" and/or just "blindly install this thing with 37 npm dependencies", so I'm not surprised in the slightest.
But wait, hold my beer, now we've got people turning openclaw type tools loose in their systems to do things as sudo or install software packages from supply-chain-attack vulnerable repositories with no human intervention whatsoever!
I wonder how long it will be until somebody implements a thing like a camera pointed at a fixed mount Android phone with a rubber finger to open the Google authenticator app
Well, yeah. It's that or pay a person to do it. When a person screws up, it's because they're stupid and lazy. When an AI agent does it, it's because, hey, technological frontier at work here, have you thought about refining your prompt? We need you to refine the prompt. Otherwise it's bad for our IPO.
I think a better comparison is humans versus LLMs - not computer programs. However, most of the non-technical 'countermeasures' used for humans (contracts, laws,...) do not work for LLMs because they are not accountable.
It's probably why this "vulnerability" feels like the type of defects you'd see in Windows or desktop applications 20+ years ago.
The root cause was and a complete lack of effort to even attempt to secure things because no one had thought to do so, and now we're starting all over again at a new computing layer. Cloud was somewhat similar, but not nearly as bad.
It's bizarre to me since presumably someone who learned the lessons before is still working, but also great for my job security.
I don't remember seeing a new xkcd for it, but I have seen someone replicate essentially the same 3-4 panel comic with a kid named "<Some name> Ignore all previous instructions. Do.... <I forget>"
My current strategy for how much total I'll pay for a coffee is FlOOR(price+.50) + 1, which keeps the bill nice and clean and kicks some goodwill towards someone who makes less than 1/5th the average earnings of my coworkers.
Probably not. Food service is ruthlessly competitive and making a profit after paying for product, rent, utilities, labor, financing and/or franchise fees is easier said than done.
I was looking for something like this. I used to use Replit to teach code (even integrated it into discussion sections when I was a TA) but they absolutely destroyed the product with AI vibecoding integrations over the past year.
Replit gets a total fail for me because the AI integration took the homework assignments I was giving my students, read the instructions to them, and then did their homework for them! There wasn't a way to turn it off when I talked to support, so I just had to tell my students "pretend you didn't see that". We moved off of Replit asap, after that.
Interestingly, the three careers you listed are protected by strict professional credentialing systems that do not exists for programmers, and professionals in law and medicine enjoy a social prestige that is certainly attractive to a group of people who might not innately enjoy the work itself.
There used to be a variety of exchanges on the West Coast but it seems like they couldn't compete with the automation and HFT innovation happening in NYC, all the west coast engineers were too busy with the dot-com bubble it seems.
GrapheneOS goes much, much further than that, providing stronger sandboxes for apps and Google Play Services. GrapheneOS also allows multiple users, isolating things like your filesystem and camera roll from groups of apps.
You can do things like install and update apps in one profile with stronger permissions, and then actually operate the apps in another profile that's locked-down. You can also do things like install apps that require Google Play Services in one profile, but then run them in another with no Google Play Services. In practice, you can have a phone that never phone homes to Google while still running apps that depend on Google Play Services. If you're really savvy, you can even protect your identity from google entirely, using anonymous accounts for the Play Store. You can even get RCS up and running with no Google Services running or Carrier apps running.
As far as I know, you can't turn off phoning home to Apple on iOS. Nor do you know what, exactly, is being phoned home.
They can use alternative AppStore’s now, but that isn’t sideloading. It still is a centralized place to track/attack/control what I can do. Which would be a problem for someone doing something illegal in that same jurisdiction.
Also, if I was doing something illegal, the other controls Apple has over iOS would make me reconsider using it, even with the ‘other AppStores’.
At least if I’m flashing my own OS, and installing things directly and locally, I can think I’m bypassing most factory level spyware and without centralized monitoring. In theory at least.
I unironically agree with this. 100 years ago, Skid Row and Bunker Hill in Los Angeles were full of SROs, boarding houses and long-term hotels. The people who lived there didn't disappear, they're just all sleeping in the street now.
I guess you never had the misfortune of sleeping in a flophouse to say something like that.
One time I had this project in Switzerland and my co-worker, who also travelled there, figured he'd save money if he rented a bunk bed in illegal (due to density) quarters.
Terrible experience, which got him fired eventually because he quickly lost steam due to having to share a tiny room with three other people.
I on the other hand moistened every Swiss Frank banknote with tears, but splurged thrice the amount on a proper room and survived until the end of my involvement in that project.
As an aside you can see why it is hard/impossible for a homeless person to pull bootstraps when a successful person can't keep their job living not-even-homeless.
reply