It's promising technology, but the tools are far from mature yet.
And as they do mature, the ramp up will decrease and their won't be any particular benefit to being an early adopter. For reasonably bright people, there's essentially no penalty to "missing out" for a while.
As often, the FOMO-afflicted are churning on stuff that just won't matter. Which is fine if they enjoy it, but isn't something the rest of us need to fret over.
100%. Imagine some talented engineer waking up from a 12 month coma. Do we honestly believe they’ll be permanently “behind” in the workforce because he wasn’t churning through LLM harnesses those past 12 months? What about 22 year old college graduates entering the workforce?
Keep abreast, don’t lose sleep, don’t sacrifice work-life balance. Help each other, especially your coworkers. The current craze seems to have created stack ranking monsters out of the whole industry.
> Eventually I’ll reach a point where I am forced to choose between the useful aspects of the model and the limiting ones instead of just picking the most capable model out there
No, the choice will be whether or not to to upgrade to "Claude Security Professional" or whatever they want to brand it as.
What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow.
And next month you'll need to add on "Claude Database Pro" or you'll just get a working (for demo purposes with dozens of db rows) but completely un indexed database schema and a refusal to optimise SQL requests.
And the month after you'll need "Claude DataScience Pro" to get any Python Pandas or NumPy code generated.
While this is a perfectly reasonable thing to expect when the models are competent enough, half the conversation on places like Hacker News are about all the times an LLM has produced garbage that was harmful to a business either by hallucinations, by deleting something critical during the work, or by hitting some endpoint way too often and denial-of-servicing it.
Right now, the software guardrails in LLMs are useful for the same kinds of reasons factories have hardware guardrails: to reduce the rate at which errors become "incidents".
Just because they sometimes delete the production database rather than sometimes spilling a thousand tons of incandescent molten metal over a factory floor, doesn't mean LLMs are safe enough to be used the way they're actually being used.
I think you're assuming too much care. Right now they haven't adopted that business model because they don't see it as a viable business model. As soon as they realize that they can lock certain categories of query behind a different subscription they will do that. We saw the same thing with streaming services and basically every other kind of online service -- small, singular subscription followed by a gold rush and then suddenly there's an upcharge for access to every other publisher's catalog of movies.
Well, I'd prefer bugfixes over exploit vectors. This will keep us honest.
With pi or better omp it would be incredibly easy to adjust the Claude system prompt so it will be easy to do what the Chinese models or gpt did. That's how the Chinese were training their models btw
Same thing with the weird push towards humanoid robots.
"They can do anything!"
Sure, once you subscribe to the $15/mo laundry package, the $25/mo lawn care package (with the $10/mo hedge trimmer upgrade), and the $10/mo dog-walking package.
When we are stabbed to death by impoverished dudes who are piloting a robot worth more than a decade of their income to do household chores for 16 hours a day, we will deserve it.
> What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow.
I don't buy this, because is predicated on staying permanently far ahead of the open weights models.
If in the future Anthropic fully stops you from doing security research, you can be sure some other provider will sell you an 'unshackled' DeepSeek v8 Pro...
> I don't buy this, because is predicated on staying permanently far ahead of the open weights models.
In my mind, that fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS, otherwise they'd change their tune really quickly, if they didn't think that was possible.
Sure, you might be able to use DeepSeek V8 Pro instead for the same purposes, but that'll hardly stop Anthropic from trying to sell bundles of use cases instead and claim it's "ethical AI", "Patriotic AI" or some marketing terms like that.
> fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS
They are just straight up delusional, no? Or at least, have a vested financial interest in maintaining said delusion until the money runs out. They have to hit the point of diminishing returns at some point...
Well, I guess that's one way to put it. Another is "dress for the job you want", startup culture typically seems to shove people in the direction of "aim big and believe in yourself, regardless of what others say" so naturally you get these companies who seem very disconnected from reality.
I'd also wager a guess that the amount of money makes people's reasoning and perspectives get very messed up as well, for better or worse.
FYI there is and been for a long time. Won't claim they're SOTA, but they exists. From the top of my head, I think Olmo (https://allenai.org/olmo) was pretty early, but been more since then too.
I agree most releases today that claim to be "open source" actually aren't, but that doesn't mean "FOSS LLMs" don't exists at all.
What? You can't give access to that kind of power to just anyone with $5,000/month.
These people should be trained and licensed before they get access. Thankfully, Anthropic has worked with regulators to develop the appropriate courses to maintain your license -- don't worry, the series is cheap when you buy all up through OT XVII. And because Anthropic has been approved as Security Overseer, we will take care of reporting back to the license bureau on our monitoring of your work to ensure you meet your ongoing license responsibilities and are able to keep your license.
Which regulators? You know, the new agency led by several of our former mid-level executives. With relationships like that, we were honored to lead the Industry Coalition that donated the final-draft regulations.
>What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow.
on the one hand agree, but on the other hand think it's reasonable in that they can then verify the person allowed to purchase access to that model is in fact a Security professional and should be allowed to do stuff like crack security.
So, supposing it's true that these models completely change the security field and humans are ~obsolete other than as pilots guiding them what to crack, you think it's reasonable that Anthropic and OpenAI should unilaterally determine who gets to be a security professional? I hope you do understand that is what you are suggesting.
Why should anyone get to determine that? Do people really want us to move to an exclusionary guild system? I thought the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
> the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
Has it? Can you prove it? I've been using computers for almost 40 years. I've seen foss-enthusiasts repeat that claim ad-nauseam, without proof. All they ave is the vague, hand-wavy, "millions of people read the code!!11".
I use both proprietary and foss software. I write both proprietary and foss software. I have not noticed a meaningful difference in security.
Then I think you haven't been paying attention. We regularly see examples of companies attempting to cover up vulnerabilities, attacking security researchers, dragging their feet on fixes, etc. Meanwhile you can easily see for yourself how long it takes various FOSS projects to get patched and often what the attitude of the devs is.
You can also take an aggregate view. Presumably skilled developers working on major projects should be expected to have similar rates of security issues. So compare CVE frequency between various FOSS and closed source projects.
A more charitable interpretation might be that a guild would not be expected to passively allow such a situation to continue to exist. I think you'd expect a guild to directly contract for the desired tools or failing that to move into production themselves.
Regulatory capture: A vendor using regulation to prevent potential competitors from producing or selling competitive goods or services
What we're discussing in this thread: A customer compelling a vendor to produce and sell a specific good or service
Do you think these are similar?
The fact that Anthropic is doing one thing (regulatory capture) is entirely irrelevant as to whether they're allowed to engage in a completely different thing (declining to sell their services/products to specific people).
> Additionally, even if there is a guild - no guild ever let a vendor pick and choose what [the guild's] capabilities were, that would be insanely dumb.
But that's not true. Again: Vendors absolutely pick and choose what their customers' capabilities are. Regardless of whether "the guild allows them to." Guilds can't force people to make or sell tools against their will – obviously.
The analog you're trying to describe doesn't exist, which is Anthropic saying nobody else can make and sell an offensive model to "the guild."
No. Customers have never been able to compel their suppliers to make or sell certain products against their will (except in collectivist regimes or like 0.00001% of natsec related instances)
This conversation gets more and more bizarre, but I’ll bite.
1) pharmaceutical companies are regularly compelled to produce specific pharmaceuticals to continue to be allowed to exist.
2) hospitals are regularly compelled to treat patients even if they can’t afford treatment, if it is a life threatening emergency.
3) car manufacturers are always compelled to produce vehicles that meet a litany of safety, weight, and efficiency standards or they can’t produce at all.
4) defense contractors are regularly compelled to produce specific defense related products for long periods of time after they would otherwise have stopped, or else.
5) even your neighborhood gas station is likely compelled to provide air refills, free or at minimal cost, or else.
6) during a wartime (command) economy, which has happened numerous times in the US alone in the last 100 years, companies have to make what their customers (the people of the United States) demand or else.
7) utilities like electric utilities regularly have to give out freebies or take losses on things as demanded by regulators, at customers behest.
Or if we go back a bit, blacksmiths, quarries, masons, etc. all had to deal with producing what the government/lord at the time wanted - often on penalty of death - during wartime, or just because they were ordered to do so.
4) The US government can compel production, but it's extremely rare
5) Not by their customers they're not, lol
6) Yep this can happen, but is extremely unusual
7) Not by their customers they're not, lol
We're illustrating how ridiculous your claim that "guilds have always been able to declare what vendors create for them" is
Now you're talking about government regulations for some reason. Even your examples of customers being able to compel production are actually examples of governments being able to compel production, and in just a few of these scenarios the government is the customer. But it's their power as governments, not their power as customers that can compel production.
As stated: you've lost the thread. You're talking about totally irrelevant stuff.
And regulations are totally not written by people elected to do so - by their customers? And many of those, the customer is the gov’t and literally doing so?
Not to mention how wild it is to operate under the assumption that they won’t give a license to an LLM that can do illegal actions to someone who shouldn’t have it. Offering it at all is an ethically dicey question.
I wish you understood that there are organizations of security professions that are not controlled by Anthropic and OpenAI and that it is a common thing that when companies of any type sell to professionals of any type it is not the companies that determine whether or not the people they sell to are professionals but membership in professional organizations.
As an example the people who sell police uniforms check that the person they are selling to is in fact a policeman (at least in the jurisdictions I have lived in, you may have had a different experience which would certainly explain what to me seems a farcical misapprehension of how modern civilization works)
I mean I just wish you understood, and really that everyone understood, that this kind of three part communication (company selling, buyer, professional organization certifying buyer) is often when buying things that are considered to have security implications.
>So, supposing it's true that these models completely change the security field and humans are ~obsolete
OK, well that strike me as a really crazy level of supposition there.
I would suppose that these models make it easier for people who want to do bad things to do bad things at scale, at the same time allowing people who want to stop bad things to help identify potential targets.
Based on my supposition I would want to stop the first and find a way of helping the second. Also because I have another supposition that the first thing is easier to do than the second.
But you obviously feel differently about this issue, no doubt because of your position of great moral stature and insight, and this no doubt prompts you to wish to me to understand things that from my position seem absolutely ludicrous.
You may already know this, but the voice engines implementations have a lot of room for performance improvement, and you could probably squeeze out even more voices, routing patterns, secondary features, etc if you tighten them up.
They're written clearly but kind of like textbook/reference implementations rather than the more aggressively fine-tuned versions you'd see in proprietary commercial synths.
You can probably coach Claude through improving them, but you generally need to do that explicitly.
Lowest hanging fruit for your design would be to have it organize control flow (and apply effective math tricks) to minimize branching and to organize data references with cpu cache more in mind.
On a more ambitious scale, working over blocks instead of samples, and using vector instructions where practical would go a long way. The latency impact of working on small blocks (i.e ~8/16/32 samples) instead of individual samples is irrelevant but the performance opportunities that open up for many algorithms is huge.
Exactly the kind of feedback I'm looking for, since this is an educational, hobby-class project for myself. I'm just treating the development of it formally as a product to stay disciplined.
Make no mistake, I'm not promoting this thing like a product, though. I'm aware there's a big gulf between this and a commercialized purpose-built product.
That said, there is a ton of headroom right now on the 8GB CM5 reference platform and I felt some of these optimizations were overkill in this pre-1.0 phase. The FX bus is already block-based, but the voice synthesis path remains pretty simple. Some of the block scaffolding for the better approach exists and I'll plan to evolve in that direction.
> Would you categorize this particular style of rhetoric to be persuasive or annoying?
Why are those the choices?
Essays are situated along countless dimensions: tone, vocabulary, author, zeitgeist, publication context, intent, subtext, relationship to other works and expressions, etc
A "good faith" reader takes all of those into consideration as they absorb the essay, and integrate it with their own intellectual situation that sits along just as many countless dimensions.
Nobody's asked to sign a notarized binding document that they wholesale agree or disagree with everything said in this essay -- or its conclusions. Nor are they obliged to have some strong reaction to it at all, let alone annoyance.
It's just one among thousands of essays about a "hot button topic", to be taken however it's personally received.
Why should Chiang have to take responsibility for making sure it's not too strongly positioned for your persomal taste. Maybe he really does see it so clearly and is simply being earnest. Maybe he enjoys the literary flourish of prose in strong language. Maybe he just wants to express something as a prose-poetic human, not maximize persuasion or non-annoyance per se.
Unless you made the mistake of having your question be something you knew about, in which case you would be forever cursed to see the janky contraption tucked behind the curtain.
Modern data centers are effectively mines, but without even the upside of supporting a local economy.
They extract local resources (land, power, grid capacity, water, etc) and sell that as compute. As a rule, the mine operators are national or multi-national firms that have no presence with which to invest the extraction profits back into the community. The local resources are harvested, processed, and sold, and then the proceeds disperse into the books of these gargantuan, far-off operators.
The only way to recover some of the profits on those local resources where they're being harvested would be with taxes or similar ongoing development obligations, but the firms specifically predate on communities too politically weak to levy those against them.
If AI isn't just an speculative bubble with gross overinvestment, they may have important value as a national economic or security interest, but they're pretty terrible and lop-sided deal for existing communities. That's why they're kept quiet.
That's like saying all fine art would be photography, all film would be CGI, or all music would be synthesized electronica.
That's not how aesthetics seem to work. Artists will make more or less good use of generative AI in their work, and it will probably seep into most media in some way or another, effecting them, but arts mostly don't get replaced and AI doesn't really offer an exception to that history.
> That's like saying all fine art would be photography
Or is it like saying most portraits will be photographs rather than paintings? There are still a lot of portraits painted (maybe even as many as the pre-camera days), but by raw numbers most portraits are created by photographers.
There will also be a longer or shorter period of time in which such technology will be abused by artists (because it's new) and at some point it will stabilize.
Like saying most pictures will be made with digital cameras.
Like saying most music will be captured and edited digitally.
You guys have an anti-AI bug and it's eating you alive and blinding you to the future that is unfolding. It's toxic and you're all 100% wrong. Your hate makes it impossible to see all of the improvements.
I've been a filmmaker for decades. This tech is the most amazing thing I've ever witnessed. And it's just getting started.
Stop being old men yelling at clouds. If you don't like it, you can continue doing things the way you're used to.
The quantity/quality tradeoff is horrific. And I speak as someone who's watched hundreds of low budget "B" movies. AI is going to allow churning out vastly more movies than anyone can ever watch. Even if some might be good, the average quality goes down, and the adverse selection problem of trying to decide if something is going to be worth committing two hours to becomes harder when you have to scroll for longer and longer distances.
There's already a bit of a fatigue with CGI and the "flat lighting" Netflix TV style. AI is just going to make that worse. Mind you, I'm old enough that I would call any movie where more than 50% of the frame for more than 50% of the runtime was never real objects and created entirely on computer "animation". It's a subtly different discipline.
But yes, there's going to be a lot of it, and it's going to rack up a lot of Netflix watch hours, in the same way that "4k crackling fireplace" does.
It seems like you have trouble handling the idea that people disagree with you. You should try listening instead of immediately jumping to being reactionary.
It isn't like saying any of those things because they are referring to recording media and not to a pipe dream invented by tech bros to attract investors so that their deeply-in-the-red company can stay afloat a little while longer. AI can't create anything.
It isn't like saying any of those things because they are referring to recording media and not to a pipe dream invented by tech bros to attract investors so that their deeply-in-the-red company can stay afloat a little while longer. AI can't create anything.
>You guys have an anti-AI bug and it's eating you alive and blinding you to the future that is unfolding
For the moment what is unfolding is a dystopia shoved down most people's throats, whether they want it or not.
And the bug is obviously the "AI bug", a foreign body recently introduced. The "bug" can't be our default previous state (no AI).
>I've been a filmmaker for decades. This tech is the most amazing thing I've ever witnessed. And it's just getting started
Hopefully it will be over soon.
>Stop being old men yelling at clouds
You know that the leading AI providers and experts in the field often discuss how AI can/might/surely will wipe us off?
Not some random guys with signs in some street corner. The main people behind it.
Now, I don't believe it's so, at least not for the reasons mentioned, like the singularity. But there are very dark results from AI in society and in many domains.
But hey, we can make pretty uncanny valley (for now) videos and special effects! No need to involve or employ humans in our art either. And the human art can be drowned in a sea of AI crap, so noone will really see our AI art either. Such amazing tech /s
Sure, and like CGI, it will change the nature of the media entirely.
Different stories shown with different treatment. With CGI, scenes zoomed out to wider shots and effects swelled even louder over lighting, intimacy, acting, etc.
Old styles didn't disappear or stop evolving entirely, of course, but the center of attention profoundly shifted and the "big" production money went with jt.
Generative AI will likely drive some kind of analogous shift in dominant film aesthetics. I don't know where, but I'm not particularly excited by it myself yet.
I don't know much about the movie industry, but I think the biggest commercial successes for early CGI was possibly Jurassic Park? I don't think it would have been nearly as good if it had all just been done with puppets.
Likewise, there's been a ton of movies since that could in theory have been done purely with SFX instead of VFX, but which is probably must better from having used VFX/CGI, titles like The Matrix, Lord of the Rings, Inception, Avatar, etc.
Obviously there was also charm in titles like E.T. and Gremlins, and I think there might still be a market for movies like that, such as the 2019 Netflix series The Dark Crystal: Age of Resistance (I loved the 1982 version as a child, though this sequel wasn't for me).
I guess the question is: will generative AI allow new movies to be made, the same way that CGI has? Or will it just be an economic shift: the same quality CGI but at a fraction of the price?
There may still be people out there who believe that AI will never be able to create CGI as good as humans can, these might be the same people who used to say that CGI can never look realistic. And if you work in VFX, I bet that you can spot a fake mountain with the slightly incorrect shadows in the distance easily, but as a simple movie watcher, I really don't see it, especially when it's only on screen for like 3 seconds.
If you look at the ten top-grossing films in the past 15 years, it's almost exclusively computer animations or SFX showcases with some token, quip-based acting thrown in every now and then (all the Marvel universe stuff).
Basically, I'm not even sure if replacing actors with AI would be stylistically perceptible for the average moviegoer. There are interesting films still being made, but almost no one is watching them, because you have the option to watch some SFX explosions and flying superheroes in Avengers VII.
Even if you do want datacenters built in your country, you probably don't want them built at the maximally explotative locations that their developers pursue.
They don't provide appreciable community value and they effectively mine limited local resources (power, grid capacity, land, water) and sell it as compute, immediately diverting the profit back out of the local economy and into very distant business accounts instead.
Builders choose their targets specifically by how well they can strong-arm weak/vulnerable communities into letting them build these mines through political influence and misrepresentation. It's bad.
What you probably want is to leverage their global market value to establish new power and grid capacity in undeveloped areas, perhaps to someday become a seed for
new communities that grow around the infrastructure development work.
But that's much more expensive than bullying and seducing a weak city council so it won't happen with regional/state/federal regulatory protections or incentives that push them away from the exploitative opportunities and towards the constructive ones.
It's promising technology, but the tools are far from mature yet.
And as they do mature, the ramp up will decrease and their won't be any particular benefit to being an early adopter. For reasonably bright people, there's essentially no penalty to "missing out" for a while.
As often, the FOMO-afflicted are churning on stuff that just won't matter. Which is fine if they enjoy it, but isn't something the rest of us need to fret over.
reply