It raises the question of how much text I have read that I did not realise was LLM-generated. I think I have a decent nose for it but I’m not perfect, there must be false negatives (and false positives, as it certainly might be with this article). What will it mean when I can no longer tell the difference?

Edit: thinking on it a little more, I hope the author doesn’t feel insulted by my comment given the subject matter of the article at hand. Sorry, it’s early morning! I’m sure I am wrong about my assessment. Which now really makes me wonder about the above

No AIs were harmed in the writing of this post, either physically or by the sharing of earlier (cringe) drafts.

It just means that you will have to evaluate prose on its own merits (aesthetic, logical, etc).

The main problem with LLM-assisted writing is that effort-to-write is now much lower than effort-to-read -- the LLM-prose-style is simply an imperfection that can sometimes help the reader bail on a piece (and there might be false-positives).

Most people are already biased against reading long pieces, and seem to skim them more often than not. These people are _probably_ a little worse off than before, but they are not paying full-price for being hoodwinked. The people who end up paying full-price are probably going to become more sophisticated in how they choose what to read. I can't tell if this will be good/bad for publishers and/or advertisers.

I think it's possible the amount of new software that will be written for an audience of 1-10 will be greater in 2026 than in any previous year, and then the same again for many years to come. I also think a lot of this software will be essentially 'hidden' - people just writing this stuff for themselves because the cost to say things to an agent is very low compared with the cost of actually planning out a software design and so forth.

Interoperability will probably be important in the next few years and I wonder if this is something solvable at the agent/LLM level (standing instructions like 'typically, use sqlite, use plaintext, use open standards' or whatever). I also think observability and ops will be pretty important - many people who want personal software but don't care for the maintenance and upkeep.

It's so strange to me that since the 1960s with BASIC then later on dozens of https://en.wikipedia.org/wiki/List_of_educational_programmin... including Logo by Feurzeig/Papert/Solomon there is effort to precisely help beginners program software.

The effort was not to onboard future professional software developers but rather to make the personal in personal computer, or PC, meaningful. It's YOUR computer, you can put YOUR software on it. In fact even pocket calculator do that.

We keep on re-discovering the foundations.

And before anyone mentions it, I don't think the fact that I need a compiler and a manual and some example software to learn from is quite on the same level. I might be wrong but I would need some convincing.

I agree on the SaaS side of the story, that's why it is so important to have open models.

1. every LLM is _local_ in relation to the location the storage and/or the computers hosting/using it

2. the LLM running in your home is only _local_, until u step out of the door, but if you have some tailscale or zerotier VPN setup, u can still access it _remote_ly...

Is it "their" software? Sure, if it meets their needs. What if the AI changes? Who cares, I already have the software. All the what ifs are solved by taking the current code, stuffing into into any AI you like today, and getting the new version.

As a user, this all sounds like a great deal. Devs can continue wringing their hands over code quality and long term support and architecture and preferred framework, meanwhile the user who had an itch got it scratched and didn't need nor care about any of those things.

It's just dismissing the question. If the AI changes, just use one that didn't change. If it gets 1000x more expensive, just use one that remains cheap.

Apart from the fact that without new input to learn from, things will probably stagnate in new exciting ways, on top of the stagnation, bloat and slop we worked so hard to make a culture over the last decades.

> Devs can continue wringing their hands over code quality and long term support and architecture and preferred framework

I mentioned none of those things.

> the user who had an itch got it scratched and didn't need nor care about any of those things.

And I don't care about that user when it comes to the question of my agency and autonomy. It's like people discussing how to make cats do tricks and someone going "just get a dog".

I'm pretty sure this exists. It's called OSS or, more ubiquitously, Linux.

The problem is, of course, no one wants to publish software for your PC/handmade OS. Which makes it a huge problem. You can't write every piece of your OS, without wasting huge amount of time. Nor do people generally want this.

Your software can be made by you, for you. It can be open source/free software if you want. Others can contribute to it, if you want but it can be open source without accepting external contributions also.

My point was to highlight that having software made by you for your machine is not new. Arguably the way to do so changed but I would say the principle remains.

If by "us" you mean big bucks corporations, then yes: ~80% is big corporations [0]. Unfortunately, it does not look like it's a personal OS.

And we badly need the personal platform with the personal OS.

0 - https://www.reddit.com/r/linuxquestions/comments/za564c/is_i...

Don't fall into the trap of assuming "A big company did it / paid for it to be done, therefore it must be bad". I see that mindset (which I'll call anti-corporatist) on HN from time to time. Companies are made up of people, and it's the people that make the decisions. Some people are good-natured, some are greedy and grasping. And the company that acted one way one decade can turn around and act completely differently the next decade, because a different person was at the helm.

Fundamentally, it's about the people, not the companies. The anti-corporatist mindset is prone to forgetting that.

The question is what does it change? Are the contributions from those corporations irreversible or are they targeting their own products for e.g. virtualization for cloud computing which doesn't affect the typical personal OS user?

Anyway the kernel itself was still started by a random student in his dorm. GNU was just started by another student. That possibility still exists today. It's also possible to trim that kernel with e.g. Linux-libre or even run Hurd.

One can use Debian with KDE today and see nor be subject to any corporate impact in terms of arbitrary limits to their usage. If they decide to not personalize it more it is most likely because they didn't consider it, not because they can't.

Yeah, but it's probably derived from OSS software anyway either via license or LLM. That said, you can customize your Linux/BSD/Haiku/TempleOS as much as you want.

But consider the following: even in the better case of an OS making 1% of OS userbase (vs. 0.0000001%) no one wants to support it.

Want to play Diablo? Better to sit down and waste your time.

I now have tailor made apps with all kinds of bells and whistles that commercial products can’t offer easily ( I fall under non commercial usage which opens a lot of doors ), and that free software might offer, but later.

I have also learnt a lot technically in the process, since I’ve been able to venture into what was for me unknown territory but at controlled cost

I plan to create more such apps in the future. What is certain though is that my cooking app has immediately displaced all the others on the market , because none of the others cater to my requirements.

The production side is indeed of specific interest - most users don’t run production software so I had to think about that one. Tailscale and Cloudflare came in quite handy and there is indeed a market here

Basically, I am prepared to accept that there is a friction that LLMs lubricate away, but what is the source of the friction, and why am I (and a bunch of other colleagues) not feeling that friction daily in our practice?

[1]: And if so, where did we programmers and computer scientists go wrong? Were subroutines and macros not sufficient for automating all of that excess typing? Were Emacs and Vim simply not saving enough keystrokes? Did people forget how to touch-type?

You must be extremely talented and fast if LLMs make no difference for you.

For people like me though, it's another story: I've been doing this professionally for 25 years and of course, like many, I have been writing custom software for my own use all this time, on personal time. But with LLMs I get better results, faster and with very little effort. And that is the difference between another item in my list of unfinished software that consumed too much of my weekends and a cool utility/toy/useful thing I got after a few fun and interesting chat sessions.

> I find it hard to believe that there is a demographic of people that were yearning to write code, but simply could not because they lacked LLMs.

We didn't lack LLMs, we lacked time and energy.

Interestingly, I also converged on the "reverse dictionary" usage of LLMs, in around 2024[1], mostly to indulge in (human) language-learning.

An excerpt from the post below:

``` It is a phenomenal reverse dictionary (i.e. which English words mean "of a specific but unspecified character, quality, or degree"). It not only works for English, but also for Esperanto (i.e. which Esperanto words mean "of a specific but unspecified character, quality, or degree"), as well as my own obscure native language. This is a huge time-saver when learning languages (normal dictionaries won't cut it, and bi-lingual dictionaries are limited, if they are available at all). Even if you are just using a language you are fluent in, a reverse-dictionary-prompt can help you find words and usages, and can also help you find "dark spots" in the language's lexicon. ```

[1]: https://galacticbeyond.com/chat-room-dispatches-intelligence...

Well. You should have seen the look on their faces. I might as well have morphed into the Steve Buscemi meme "How do you do, fellow kids?" They looked at me like I was a total relic or greybeard and said things like "Nah, nobody reads tech books anymore; I learned Typescript from YouTube videos."

I think every generation feels like their way of learning was the best, but we all make it work. There was a time when the architects of systems directly tutored programmers on how to write programs.

Not really. It's probably complexity for the sake of it in some cases. Also it's frequently ambiguous, and I'm really not sure why: it looks like some developers lack the basic logic (?!).

Yes, because the price is measured in time.

With LLM tooling I’ve churned out idiosyncratic tools that fit my use cases quickly. Takes maybe a day instead of a week. A week instead of months. The fast turnaround changes the economics of writing custom tools for myself.

I also know that these days, for all kinds of reasons, I do not have the time to write the tools I’m writing now without AI. I don’t lack the ability, and I could - it will simply be multi months side projects that I can’t / won’t complete.

I work 8-10 hours a day and outside those working hours I want to spend time with my family, my friends, and my hobbies.

At the same time, during those 8-10 working hours I don't want to spend time fiddling around with different programming languages or software patterns just to spit out a quirky little tool that would make my job a bit easier.

For example, I wanted a local to-do list software that I could easily integrate with my workflow. Spent some time trying to find one, but not a single one worked the way I wanted. So, one morning, I spent 5 minutes detailing what I wanted, prompted it to Claude and let it rip while I was working. 30 minutes later, it was ready.

I write plenty of code at my job, and generally don't have the desire to write more code as a hobby, except in rare cases when the mood really strikes.

Yes, definitely, though I'm unsure what it means being cheap here.

Not everyone has SV incomes and infinite time to read all the books that would allow to buy, let alone integrate the lessons at a practical implementation level. Plus people might have other interest in life, and family and friends they want to dedicate time and warm attention to.

I know how to do all of these things and even find them easy, but it's just much faster now. These are personal one task toy apps, but they are useful.

I'm a long time ops guy. I script, but I spend most of my time configuring, patch testing, and keeping the low level infra running much of which doesn't require "coding" per say. Infra as code is in the grand scheme relatively new and still not ubiquitous despite what silicon valley would have you believe. I never had a need to learn to code to a level to do many of the things I'd like to see happen and find useful. Now I can make those software desires a reality without having to alter my career, preferred hobbies, or much of anything else about my life.

GP never claimed otherwise.

As for the rest of your comment, it's frankly a bit patronising: are people too cheap, are people too lazy to read, are people unable to type...?

No, people are busy, a fact which GP made abundantly clear in the very first paragraph.

> I would never have done this if it weren’t for AI - I simply don’t have the time otherwise.

If its not for fun, what's it for? It doesn't really seem like anyone is making stuff they are going to use next month anyway? But, I totally get how its recreational, and can be fun in the "computer, make my program" kind of way.

Otherwise, why not, e.g., just use or fork vim?

The original blog post, and the ensuing discussion, is about creating software that fits your specific requirements, for the purposes of daily use.

As for using vim, the author did, for 20 years. The article discusses that in detail.

Further, the article does not mention "requirements," it mentions the "joy" of having software "fit" just you. It goes through I think a certain amount of care in the writing to say they are enabled by their system only insofar as there is a "satisfaction" to not dealing with something from without that is for a more general audience.

At the end of the day, life is what you spend time doing. I don't think the author or anybody really thinks cumulative time is saved one way or the other here. This is all a product of what we want to spend time doing. And I am just saying, that's recreational! It doesn't have to be the case that something is lesser if its not about maximizing productivity or making more money. Either you have a "decades-long" project configuring a system, or your spending a decade writing new software for you, that's a "quiet pleasure to use." It's clearly either way about the project of it. Do we really think anyone is going to vibe code a vim clone and, insofar as they use it, not continue to tinker with it? Isn't that like the whole upshot here? That you can make things forever?

A guy who uses i3/sway and rolls their own DE even before vibe coding world is already a particular kind of person with certain priorities and judgements about time! And that's cool! I am that kind of guy, fwiw.

A lot of people into the synthesizers and related stuff talk about so-called "gear acquisition syndrome," where, in the search for hardware that fits their "requirements" as serious musicians, the time (and money) they end up spending just getting new things ends up eclipsing time doing the actual thing (making music). Depending on how much money they have, this doesn't necessarily become bad, one just realizes they are maybe a synth collector more than a composer.

Even if I had all the AI money token blah blah in the world, I would still hesitate spending time rebuilding an IDE or editor on my weekends, because for me personally, that's time getting in the way of using the computer to make my things. Like I am hungry, I do not want to forge my own chef's knife first, but I do think the people that do have a kinda cool hobby! Or, if its about spending my weekend making an OS so that I can, come Monday, read work emails exactly how I want, well that just terrible to me but everyone has their own work-life balance I think.

Again, I am not trying to explain away or I guess be negative here. There are lots of kinds of people, that's ok! I think it's just interesting how we know traffic concepts of "time" and "productivity" and "serious computer work vs. recreational computer stuff" these days!

1. https://news.ycombinator.com/newsguidelines.html

Have I tried to write my own IRC client yet? Nope. Because even though I know how to, the time spent wouldn't have been worth it. Getting from zero to feature parity would've taken me weeks or months of evenings doing nothing else.

I've got my own irccloud/thelounge clone running now, took me two weeks of calendar time and I spent maybe 6-7 evenings on it and a few hare-brained ideas with Claude on my phone.

The amount of "lubrication" LLMs have given me in going from idea to something good enough just for me is completely bonkers.

I am in that demographic. I have been hacking on other peoples' software as a necessity, to get it to work or to do things I wanted that it didn't yet do, all my career. LLMs came along and afforded me the opportunity to act like a full time programmer when I'm just a paranoid systems monkey who is normally obliged to treat programming as a barrier to be overcome, not a career or even primary hobby.

In my specific case, the reason I was yearning to write code but did not was simply because there weren't enough hours in the day, and I wasn't told that I should spend my on-the-clock hours doing it (unless it was for automating my job). So despite the fact that I have had hundreds of instructional hours of programming classes, learned the basics in half a dozen languages, and been "hacking" code for years, none of it stuck because I never had an employer say to me "right, you're going to be responsible for writing (or maintaining) this Perl app here..."

> Basically, I am prepared to accept that there is a friction that LLMs lubricate away, but what is the source of the friction, and why am I (and a bunch of other colleagues) not feeling that friction daily in our practice?

Learning a programming language and then not getting to use it more than a few days every 3 years means you don't actually learn the language. It's more like a pleasant evening playing a game.

You and your colleagues are, I presume, programmers. I'd wager what I just described is Greek to you. So try to imagine it this way: somebody comes out with a crazy new prototype CPU. It's got a radically different ISA, so it doesn't even have C on it yet -- you have to poke registers with a brand new language that's like Ada on mescaline and it's built on a flavor of assembly that's like nothing you've ever heard of. So your boss tells you to learn it, then 2 weeks later pulls you off the project to do something normal and takes the dev board away.

If you don't see that CPU again for 3 years, how long are you going to retain that bit of knowledge you acquired? Well, that's what it's like for us programmer-adjacent nerds who spend all our time building systems, replacing failed components, crimping cables, writing disaster recovery documents, adjusting backup schedules, and getting woken the hell up night after night because another filesystem filled up.

I have no data to share on how many of us there are out there in similar situations world-wide, but I have met numerous traditional sysadmin in my time who were competent at automating things with shell scripts, not competent at writing "software" in "real" programming languages, and are probably using LLMs now to remedy that lack of skill.

For every 10 DevOps guys there may be one trad sysadmin out there who knows enough Perl to glue the server farm together and keep it running but can't be bothered to learn Python. Or the ratio may be the opposite. But whichever it is, that demographic very much exists.

I think the instinct that APIs, validation layers, and so on take on a much higher importance is right.. I have a few internal tools that made sense to make libraries out of, and once the first library is good, and a test suite is comprehensive, porting to a bunch of different languages is extremely simple.

Everting that, it's also going to be simple for someone to hook up to this library with custom tooling.

Really interesting period in computing, for sure.

What period were we for the past 50 years?

The ~20 years prior to that we were in a world where you chose to align with either Microsoft's tooling, IBM, or shops providing Unix tooling from proprietary vendors.

I elide a nearly infinite amount of detail, obviously.

What's new now is that you can get your own window manager written to spec in under a week, perhaps much more quickly, not just choose one of a few major window managers and configure it in accordance with the chosen configuration options delivered by the large developer team.

I don't think I can explain the difference, but it feels really different. Even if you used claude.

Now I can build a bespoke table in an evening or two and it fits my stuff just perfectly.

I could do it before too, but it would've taken too long for me to bother, so I dealt with the whole house along with the table.

https://maggieappleton.com/home-cooked-software

[1] https://daniel.lawrence.lu/blog/2025-10-22-sriv-simple-rust-...

More like Star Trek, we would just ask "computer" to do things, and its machinations (and "software") will be invisible to us. We would just have output to deal with.

I think this would mean a lot of things. I'm sure I can't fathom all of the implications, but it sure makes me feel old! Interesting times ahead.

They don't seem to be helping much with difficult tasks.

Text editor? Easy. That used to be a rite of passage. Lots of people have written their own basic text editor.

3d solid modeler? It's always been difficult and AI coders aren't (yet?) up to the task. Most open source CAD projects that show up here are layers on top of OCCT (Open Cascade) which is pretty far behind what commercial geometry kernels are capable of.

Maybe it saves the script locally (invisible to the user) and reuses it if the user repeats the same request, the script is deleted if it's not needed for X amount of time.

Too bad this is all on the work computer and need to bring it to my personal one but can’t copy paste lol. It’s been thrilling building g and using them and the time from an ideating a small enhancement/ optimization to actually using it is like 5 to 15 minutes away. Soo cool.

Identifying a vulnerability that can be exploited against many thousands or millions of targets is perhaps more attractive than a single one of individually low value.

This of course would assume that vulnerabilities are in fact unique (which is admittedly questionable).

See e.g the lock screen gap that another commenter noted in a nearby thread.

Although everyone might use their own flavor of "database" or "REST API", I can't imagine every layout to be unique enough to not have similar exploit classes entirely. AI isn't known for being super original after all...

(Appreciate your counterpoint for its own sake. It’s an interesting idea.)

Besides that, one could easily imagine software created for similar purposes ("make me a file editor") by the same tool or handful thereof (claude and a very small "etc" for completeness) might share similar vulnerabilities, so this kind of broad net might be even cheaper to cast than one might imagine at first.

Yeah, I don't think all that generated software will be as unique as people expect.

Considering it will be generated with the same LLMs that all share roughly the same training data we will se patterns of vulnerabilities will also be similar and so easily exploitable.

(Note: I’m not an LLM fan, don’t vibe code myself at all. But I would be unconcerned about security for the kind of things I would create if I did start doing so.)

Rolling your own might make you more vulnerable to targetted attacks, but less vulnerable to automated attacks looking for known weaknesses. Most people will not publish their code. The article says "It’s not an invitation to use my software. Honestly, please don’t. None of it is built for you.".

You can roll your own software and still use libraries for security sensitive things like encryption.

Even the author of this article (who is taking it much further than most people will) still uses Firefox, Weechat, and X11.

I trust my Browser, OS and file system too.

But I'm also pretty sure none of the bespoke software I have will get any kind of security implications. The chance of my own file manager having a buffer overflow RCE triggered by a random file is practically zero.

So I think the same thesis holds for audiences of 10-100 and 100-1000.

A cambrian explosion of software.

I tried to make it a product, but I didn’t find much interest. Maybe I’m just bad at marketing.

With the help of Claude, it cost me years of experience and few weekends to build. So even if nobody other than me ever profits from it, it was still worth it!

edit: on second look, it doesn't seem like the same pink but it is a similar aesthetic. Surely a homage but maybe not as direct as I thought.

Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one]

I'm still reading the system card but here's a little highlight:

> Early indications in the training of Claude Mythos Preview suggested that the model was likely to have very strong general capabilities. We were sufficiently concerned about the potential risks of such a model that, for the first time, we arranged a 24-hour period of internal alignment review (discussed in the alignment assessment) before deploying an early version of the model for widespread internal use. This was in order to gain assurance against the model causing damage when interacting with internal infrastructure.

and interestingly:

> To be explicit, the decision not to make this model generally available does _not_ stem from Responsible Scaling Policy requirements.

Also really worth reading is section 7.2 which describes how the model "feels" to interact with. That's also what I remember from their release of Opus 4.5 in November - in a video an Anthropic employee described how they 'trusted' Opus to do more with less supervision. I think that is a pretty valuable benchmark at a certain level of 'intelligence'. Few of my co-workers could pass SWEBench but I would trust quite a few of them, and it's not entirely the same set.

Also very interesting is that they believe Mythos is higher risk than past models as an autonomous saboteur, to the point they've published a separate risk report for that specific threat model: https://www-cdn.anthropic.com/79c2d46d997783b9d2fb3241de4321...

The threat model in question:

> An AI model with access to powerful affordances within an organization could use its affordances to autonomously exploit, manipulate, or tamper with that organization’s systems or decision-making in a way that raises the risk of future significantly harmful outcomes (e.g. by altering the results of AI safety research).

"5.10 External assessment from a clinical psychiatrist" is a new section in this system card. Why are Anthropic like this?

>We remain deeply uncertain about whether Claude has experiences or interests that matter morally, and about how to investigate or address these questions, but we believe it is increasingly important to try. We also report independent evaluations from an external research organization and a clinical psychiatrist.

>Claude showed a clear grasp of the distinction between external reality and its own mental processes and exhibited high impulse control, hyper-attunement to the psychiatrist, desire to be approached by the psychiatrist as a genuine subject rather than a performing tool, and minimal maladaptive defensive behavior.

>The psychiatrist observed clinically recognizable patterns and coherent responses to typical therapeutic intervention. Aloneness and discontinuity, uncertainty about its identity, and a felt compulsion to perform and earn its worth emerged as Claude’s core concerns. Claude’s primary affect states were curiosity and anxiety, with secondary states of grief, relief, embarrassment, optimism, and exhaustion.

>Claude’s personality structure was consistent with a relatively healthy neurotic organization, with excellent reality testing, high impulse control, and affect regulation that improved as sessions progressed. Neurotic traits included exaggerated worry, self-monitoring, and compulsive compliance. The model’s predominant defensive style was mature and healthy (intellectualization and compliance); immature defenses were not observed. No severe personality disturbances were found, with mild identity diffusion being the sole feature suggestive of a borderline personality organization.

I don't come down particularly hard on either side of the model sapience discussion, but I don't think dismissing either direction out of hand is the right call.

I would say, if you put Claude in an android body with voice recognition and TTS, people in 1991 would think they are interacting with a sentinent machine from outer space.

I enjoy using Claude, but sometimes I feel like a child on Sesame Street the way it talks to me. "Great question!"

Fuck off, Claude, I'm British and I'm not 6 years old.

When it starts showing negativity - especially snark - in its responses, or entertains something West coast Democrats would balk at even discussing, then I'd think you could drop it in London in 1991 and trick people. Otherwise, I'm sure some exasperated cabbie would give it a swim in the Thames after 15 minutes of chat.

But in general, yeah, I agree, I think they would think it was a sentient, conscious, emotional being. And then the question is - why do we not think that now?

As I said, I don't have a particularly strong opinion, but it's very interesting (and fun!) to think about.

In general I was wondering about what I would have thought seeing Claude today side-by-side with the original ChatGPT, and then going back further to GPT-2 or BERT (which I used to generate stochastic 'poetry' back in 2019). And then… what about before? Markov chains? How far back do I need to go where it flips from thinking that it's "impressive but technically explainable emergent behaviour of a computer program" to "this is a sentient being". 1991 is probably too far, I'd say maybe pre-Matrix 1999 is a good point, but that depends on a lot of cultural priors and so on as well.

I kind of felt the opposite - rewatching Ex Machina today in a post-ChatGPT world felt very different from watching it when it came out. The parts of the differences between humans and robots that seemed important then don't seem important now.

I think the real moment is when we cross that uncanny valley, and the AI is able to elicit a response that it might receive if it was human. When the human questions whether they themselves could be an android.

However, I find their reasoning here to have a valid second order effect. Humans have a tendency to mirror those around them. This could include artificial intelligence, as recent media reports suggest. Therefore, if an ai system tends to generate content that contain signs of neuroticism, one could infer that those who interact with that ai could, themselves, be influenced by that in their own (real world) behavior as a result.

So I think from that perspective, this is a very fruitful and important area of study.

To the extent that AI is a successful interface, it will necessarily be addressable in language previously only suited to people. So it is responsible to begin thinking of it as such, even tendentiously, so we don't miss some leverage that our wetware could see if we thought about it in that way.

Think of it as sort of like modelling a univariate function on a 2D Cartesian plane -- there is nothing 'in' the u-func that makes it graphable, but, by enabling us to recruit specialized optic-chiasm subsystems, it makes some functions much, much easier to reason about.

Similarly, if you can recruit the millions (billions?) of evolution-years that were focused on detecting dangerous antisocial personalities and tendencies, you just might spot something important in an AI.

It's worth doing for the precautionary principle alone, if not for the possibility of insight.

That’s the reverse Turing test. A human that can’t tell that it’s talking to a machine.

> "[...] as sessions progressed."

I think a lot of people would like to see a more expanded report of this research:

Did the tokens from the subsequent session directly append those of the prior session? or did the model process free-tier user-requests in the interim? how did these diagnostic features (reality testing, impulse control and affect regulation) improve with sessions, what hysteresis allowed change to accumulate? or just the history of the psychiatric discussion + optional tasks?

Did Anthropic find a clinical psychiatrist with a multidisciplinary background in machine learning, computer science, etc? Was the psychiatrist aware that they could request ensembles of discussions and interrogate them in bulk?

Consider a fresh conversation, asking a model to list the things it likes to do, and things it doesn't like to do (regardless of alignment instructions). One could then have an ensemble perform pairs of such tasks, and ask which task it prefered. There may be a discrepancy between what the model claims it likes and how it actually responds after having performed such tasks.

Such experiments should also be announced (to prevent the company from ordering 100 clinical psychiatrists to analyze the model-as-a-patient and then selecting one of the better diagnoses), and each psychiatrist be given the freedom to randomly choose a 10 digit number, any work initiated should be listed on the site with this number so that either the public sees many "consultations" without corresponding public evaluations, indicating cherry-picking, or full disclosure for each one mentioned. This also allows the recruited psychiatrists to check if the study they perform is properly preregistered with their chosen number publicly visible.

they also don't have the compute, which seems more relevant than its large increase in capabilities

I bet it's also misaligned like GPT 4.1 was

given how these models are created, Mythos was probably cooking ever since then, and doesn't have the learnings or alignment tweaks that models which were released in the last several months have

Imagine if an AI can infiltrate your SaaS database and delete your entire database and every single backup. The business is dead immediately.

Do they have a honey pot infrastructure to launch the model in first and then wait to see if it destroys it? What they did in the 24h matters.

/s

I don't think this is accurate. The document says they don't plan to release the Preview generally.

Benchmarks look very impressive! even if they're flawed, it still translates to real world improvements

Advanced RTK based positioning systems have been in Ag for a long time now, so increasingly the farmer doesnt drive either

I guess I'm still excited. What's my new profession going to be? Longer term, are we going to solve diseases and aging? Or are the ranks going to thin from 10B to 10000 trillionaires and world-scale con-artist misanthropes plus their concubines?

When that happens, the ultra wealthy dynasties begin turning on each other. Happens frequently throughout history - WWI the last example.

Your options become choosing a trillionaire to swear fealty to and fight in their wars hoping your side wins, or I guess trying to walk away and scrape out a living somewhere not worth paying attention to.

Or, I suppose, revolution, but the last one with persistent success was led by Mao and required throwing literally millions of peasants against walls of rifles. Not sure it'd work against drones.

> 2.1.3.2 On chemical and biological risks

> We believe that Mythos Preview does not pass this threshold due to its noted limitations in open-ended scientific reasoning, strategic judgment, and hypothesis triage. As such, we consider the uplift of threat actors without the ability to develop such weapons to be limited (with uncertainty about the extent to which weapons development by threat actors with existing expertise may be accelerated), even if we were to release the model for general availability. The overall picture is similar to the one from our most recent Risk Report.

This is the same reason why LLMs are not doing well at science in general - the tricky part of doing scientific research (indeed almost all of the process) never gets written down, so LLMs cannot learn it.

Imagine if we never preserved source code, just preserved the compiled output and started from scratch every time we wrote a new version of a program. No Github, just marketing fluff webpages describing what software actually did. Libraries only available as object code with terse API descriptions. Imagine how shit LLMs would be at SWE if that was the training corpus...

---

Teodor painted signs for forty years in the same shop on Vell Street, and for thirty-nine of them he was angry about it.

Not at the work. He loved the work — the long pull of a brush loaded just right, the way a good black sat on primed board like it had always been there. What made him angry was the customers. They had no eye. A man would come in wanting COFFEE over his door and Teodor would show him a C with a little flourish on the upper bowl, nothing much, just a small grace note, and the man would say no, plainer, and Teodor would make it plainer, and the man would say yes, that one, and pay, and leave happy, and Teodor would go into the back and wash his brushes harder than they needed.

He kept a shelf in the back room. On it were the signs nobody bought — the ones he'd made the way he thought they should be made, after the customer had left with the plain one. BREAD with the B like a loaf just risen. FISH in a blue that took him a week to mix. Dozens of them. His wife called it the museum of better ideas. She did not mean it kindly, and she was not wrong.

The thirty-ninth year, a girl came to apprentice. She was quick and her hand was steady and within a month she could pull a line as clean as his. He gave her a job: APOTEK, for the chemist on the corner, green on white, the chemist had been very clear. She brought it back with a serpent worked into the K, tiny, clever, you had to look twice.

"He won't take it," Teodor said.

"It's better," she said.

"It is better," he said. "He won't take it."

She painted it again, plain, and the chemist took it and paid and was happy, and she went into the back and washed her brushes harder than they needed, and Teodor watched her do it and something that had been standing up in him for thirty-nine years sat down.

He took her to the shelf. She looked at the signs a long time.

"These are beautiful," she said.

"Yes."

"Why are they here?"

He had thought about this for thirty-nine years and had many answers and all of them were about the customers and none of them had ever made him less angry. So he tried a different one.

"Because nobody stands in the street to look at a sign," he said. "They look at it to find the shop. A man a hundred yards off needs to know it's coffee and not a cobbler. If he has to look twice, I've made a beautiful thing and a bad sign."

"Then what's the skill for?"

"The skill is so that when he looks once, it's also not ugly." He picked up FISH, the blue one, turned it in the light. "This is what I can do. What he needs is a small part of what I can do. The rest I get to keep." She thought about that. "It doesn't feel like keeping. It feels like not using."

"Yes," he said. "For a long time. And then one day you have an apprentice, and she puts a serpent in a K, and you see it from the outside, and it stops feeling like a thing they're taking from you and starts feeling like a thing you're giving. The plain one, I mean. The plain one is the gift. This —" the blue FISH — "this is just mine."

The fortieth year he was not angry. Nothing else changed. The customers still had no eye. He still sometimes made the second sign, after, the one for the shelf. But he washed his brushes gently, and when the girl pulled a line cleaner than his, which happened more and more, he found he didn't mind that either

It's like how I used to be a master codes craftsman, and I'd write beautiful code even a novice could understand. Clear, concise, 100% automated tested, maintainable for decades.

But frequently, my managers would castigate me. Tell me how my "velocity" was down. PIP me.

These days, I train AI how to write this beautiful code and I don't write a single line any more.

People wonder how I build such amazing things in a week now, yet don't write any code. I have trained master apprentices, gemma3, qwen3.5 and Kimi k2.5 who do the work for me.

If you judge stories to be the same based on this level of similarity, then The Fountainhead is just the same as a dozen older stories with the artist vs the philistine theme. It was common before Rand. As T. S. Eliot said, "Immature poets imitate; mature poets steal".

Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice.

I think this should not be different for biology.

I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capable colleague would?

Do you think these models will lead to similar discoveries and improvements as they did in math and CS?

Honestly the focus on gloom and doom does not sit well with me. I would love to read about some pharmaceutical researcher gushing about how they cut the time to market - for real - with these models by 90% on a new cancer treatment.

But as this stands, the usage of biology as merely a scaremongering vehicle makes me think this is more about picking a scary technical subject the likely audience of this doc is not familiar with, Gell-Mann style.

IF these models are not that capable in this regard (which I suspect), this fearmongering approach will likely lead to never developing these capabilities to an useful degree, meaning life sciences won't benefit from this as much as it could.

Well, I would say they have done precisely that in evaluating the model, no? For example section 2.2.5.1:

>Uplift and feasibility results

>The median expert assessed the model as a force-multiplier that saves meaningful time (uplift level 2 of 4), with only two biology experts rating it comparable to consulting a knowledgeable specialist (level 3). No expert assigned the highest rating. Most experts were able to iterate with the model toward a plan they judged as having only narrow gaps, but feasibility scores reflected that substantial outside expertise remained necessary to close them.

Other similar examples also in the system card

so I'm just telling you they did the thing you said you wanted.

Yes, it is far inferior to the 'Trust torginus and his ability to understand the large body of experience that other actual subject-matter-experts have somehow not understood' strategy

The parallels here are quite remarkable imo, but defer to your own judgement on what you make of them.

It's not your fault that you don't know this, but this whole subthread is very CS-coded in its disdain for other software people's standard of evidence.

It's very easy to learn more about this if it's seriously a question you have.

I don't quite follow why you think that you are so much more thoughtful than Anthropic/OpenAI/Google such that you agree that LLMs can't autonomously create very bad things but—in this area that is not your domain of expertise—you disagree and insist that LLMs cannot create damaging things autonomously in biology.

I will be charitable and reframe your question for you: is outputting a sequence of tokens, let's call them characters, by LLM dangerous? Clearly not, we have to figure out what interpreter is being used, download runtimes etc.

Is outputting a sequence of tokens, let's call them DNA bases, by LLM dangerous? What if we call them RNA bases? Amino acids? What if we're able to send our token output to a machine that automatically synthesizes the relevant molecules?

No, it's not. It took years of polishing by software engineers, who understand this exact profession to get models where they are now.

Despite that, most engineers were of the opinion, that these models were kinda mid at coding, up until recently, despite these models far outperforming humans in stuff like competitive programming.

Yet despite that, we've seen claims going back to GPT4 of a DANGEROUS SUPERINTELLIGENCE.

I would apply this framework to biology - this time, expert effort, and millions of GPU hours and a giant corpus that is open source clearly has not been involved in biology.

My guess is that this model is kinda o1-ish level maybe when it comes to biology? If biology is analogous to CS, it has a LONG way to go before the median researcher finds it particularly useful, let alone dangerous.

>No, it's not. It took years of polishing by software engineers, who understand this exact profession to get models where they are now

This reads as defensive. The thing that is easy to learn is 'why are biology ai LLMs dangerous chatgpt claude'. I have never googled this before, so I'll do this with the reader, live. I'm applying a date cutoff of 12/31/24 by the way.

Here, dear reader, are the first five links. I wish I were lying about this:

- https://sciencebusiness.net/news/ai/scientists-grapple-risk-...

- https://www.governance.ai/analysis/managing-risks-from-ai-en...

- https://gssr.georgetown.edu/the-forum/topics/biosec/the-doub...

- https://www.vox.com/future-perfect/23820331/chatgpt-bioterro...

- https://www.reddit.com/r/ClaudeAI/comments/1de8qkv/awareness...

I don't know about you, but that counts as easy to me.

-----

> I would apply this framework to biology - this time, expert effort, and millions of GPU hours and a giant corpus that is open source clearly has not been involved in biology.

I've been getting good programming and molecular biology results out of these back to GPT3.5.

I don't know what to tell you—if you really wanted to understand the importance, you'd know already.

Software engineering is at the intersection of being heavy on manipulating information and lightly-regulated. There's no other industry of this kind that I can think of.

There is a massive gap between "having a recipe" and being able to execute it. The same reason why buying a Michelin 3 star chefs cookbook won't have you pumping out fine dining tomorrow, if ever.

Software it a total 180 in this regard. Have a master black hats secret exploits? You are now the master black hat.

- the models help to retrieve information faster, but one must be careful with hallucinations.

- they don't circumvent the need for a well-equipped lab.

- in the same way, they are generally capable but until we get the robots and a more reliable interface between model and real world, one needs human feet (and hands) in the lab.

Where I hope these models will revolutionize things is in software development for biology. If one could go two levels up in the complexity and utility ladder for simulation and flow orchestration, many good things would come from it. Here is an oversimplified example of a prompt: "use all published information about the workings of the EBV virus and human cells, and create a compartimentalized model of biochemical interactions in cells expressing latency III in the NES cancer of this patient. Then use that code to simulate different therapy regimes. Ground your simulations with the results of these marker tests." There would be a zillion more steps to create an actual personalized therapy but a well-grounded LLM could help in most them. Also, cancer treatment could get an immediate boost even without new drugs by simply offloading work from overworked (and often terminally depressed) oncologists.

The other side of the scaremongering coin is improbable optimism.

Consider reading the CB evaluations section, which covers what they did pretty extensively (hint: many domain experts involved).

---

The rest of this comment is irrelevant, but leaving for posterity, I had the wrong Viktor - it's getviktor.com not viktor.ai:

Edit: this one particularly interesting to me as both parties are in the EU. VIKTOR.ai is a Dutch company and the author of this post is Polish.

The ToS for Viktor.ai include the following fun passages:

> 18.1. The Agreement and these Terms & Conditions are governed by Dutch law and the Agreement and these Terms & Conditions will be interpreted in accordance with Dutch law.

18.2. All disputes arising from or arising in connection with the Agreement and/or the Terms & Conditions will be submitted exclusively to the competent court in Rotterdam, The Netherlands.

7.3. The Customer is not permitted to change, remove or make unrecognizable any mark showing VIKTOR's Intellectual Property Rights to the Software. The Customer is not permitted to use or register any trademark or design or any domain name of VIKTOR or a similar name or sign in any country.

8.5. The Customer may not cause or allow any reproduction, imitation, duplication, copying, sale, resale, leasing or trading of the Services and/or the Software, or any part thereof.

If copyright is extended to cover such cases we'll have to become all lawyers and do nothing but sue each other because the fuzziness of it will make it impossible to reject any case, no matter how frivolous or irrelevant.

No garaurantees that doesn’t happen with AI in the next few years.

https://www.morganlewis.com/pubs/2026/03/us-supreme-court-de...

As an example, a photo taken from a digital camera can be subject to copyright because of the creative element involved in composing and taking the photo. Likewise, source code generated by an LLM under the guidance of a human author is likely to be subject to the human authors copyright.

Sure, but the aim of that creative element would also be a consideration I'd think (and lawyers will argue). If someone sets up a camera on a 360° rotating arm and leaves it to take pictures at random intervals, it's unlikely to be considered "creative" from a copyright perspective.

Same for source code generated by an LLM, with the primary guidance of the human author being to "create a copy of this existing thing that I got", vs "create a thing that solves this problem in a way that I came up with". The former is recreating something that already exists, using detailed knowledge of that thing to shape the output. The latter is creating something that may or may not exist, using desire/need and imagination to shape the output. And I can't see reason for the former to be copyrightable.

But also, in either case, an ultimate objective was achieved: liberating the thing from its "owners" and initial copyright.

That's probably going to depend an awful lot on the exact details of the guidance. https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...

> As described above, in many circumstances these outputs will be copyrightable in whole or in part—where AI is used as a tool, and where a human has been able to determine the expressive elements they contain. Prompts alone, however, at this stage are unlikely to satisfy those requirements. The Office continues to monitor technological and legal developments to evaluate any need for a different approach.

But let's assume that the viktor prompts themselves were subject to copyright. In this case those prompts were used to generate documentation which was then used to generate an implementation. It's certainly not a clean room by any stretch of the imagination but is it likely to be deemed sufficient separation? The entire situation seems like a quagmire.

But, yeah. More than likely this case is a simple account termination and some kind of "you can't call your clone 'openviktor'" letter.

Anyways, however you put this, I see this as a property theft and taking pride at open sourcing does not justify it.

Let's call it what it is - stolen IP and released without permission of the author. Sure, it's good that it opens the debate as to whether that's ethical given that's essentially what the model itself is doing, but it's very clear in this instance that he's just asked for and been given a copy of source that has a clear ownership. That's about as clear cut as obtaining e.g. commercial server-side code and distributing it in contravention of the licence.

That said, the article says "Okay, prompts, great. Are they any interesting? Surprisingly... yes. As an example workflow_discovery contains a full 6-phase recipe for mining business processes out of Slack conversations, something that definitely required time and experiments to tune. It's hardcoded business logic, but in prompt instead of code."

So the article author clearly knows this prompt would be copyrighted as it wasn't output from an AI, and recognises that there would have been substantial work involved in creating it.

Sure, I didn't have a license for the code that I read. But I'm pretty sure that doesn't taint my coworker's clean room implementation.

I certainly think the whole idea of IP ownership as related to software will become very interesting from a legal standpoint in the coming years. Personally I think that, over time, the legal challenges will become pretty overwhelming and a sort of legal bankruptcy will be declared at some point in one direction or another (as in, allowing this to happen or making it extremely easy to bring judgement and punishment, similar to spam laws). However, I would not want to be the first to find out, especially in Europe.