It's also not $1,500 per month per engineer. It's that per month per engineer per tool. Which means it could easily be at least $3,000 (Claude Code and Cursor) or $4,500 if Codex was also an option on top of those two.
And as you have written on your blog it's a soft cap that can be exceeded with justification.
No reason to doubt his journalistic integrity? He's not a journalist for starters. He's a PR flack who does PR for AI startups on the side while blogging on substack. There is every reason to doubt his journalistic integrity.
The PR-thing was always openly communicated by him and is not some secret or gotcha. It's essentially "fleecing the boosters", which I fully approve of and do similarly myself.
I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.
It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.
So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.
He was right about the cost changes, which he predicted quite some time ago. People shouted at him that he was making it all up - yet it was correct.
He was also right about AI-video and sora in particular being a fundamentally flawed idea.
He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it.
Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that.
The general direction seems correct to me.
He's not a technical guy and does not have the knowledge to critique models on a factual basis. I do wish he'd just focus on the stuff he _does_ know about, which is the financial side of things.
He is a much needed counterweight to the unhealthy hype going around, imho.
> He was also right about AI-video and sora in particular being a fundamentally flawed idea.
He specifically predicted that AI videos have plateaued in 2024 which is egregiously wrong.
> He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it.
He specifically predicted that accuracy won't increase but accuracy has increased over the time significantly to the point where you can't get it to say anything inaccurate using the reasoning models.
> Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that.
The whole Uber skepticism is a good point because all of those people were wrong and Uber is profitable now.
You didn't address my other criticisms - he claimed that revenue would drop in 2024 and it skyrocketed. He claimed that users weren't interested in ChatGPT but now it has a billion users (6x jump).
Because it doesn't. Not for the tasks where using Opus instead of a lower tier model is appropriate, at any rate. Benchmarks show this, as do revealed preferences of actual users. To believe that Qwen is as capable as Opus at 1/20 the cost you have to believe that every person who does not make the choice to use Qwen over Opus for a given task is some mix of ignorant or delusional. This is certainly an opinion you can hold about other engineers, but it's definitely a questionable one at best.
The benchmarks between the two are close and the engineers that have used both (like myself) can attest that the differences aren't so wide as you might believe.
I'd say that yes, ignorance plays a role here because a decent number of engineers are looking strictly at the benchmarks and choosing Opus just for that reason.
But I'd also say that a major factor for Opus use is because Opus is being purchased for the engineers by their employers. They don't get to pick which models they are using.
I find myself rarely reaching for Opus nowadays, it's just too slow. I assume there are tricky use-cases where it's really useful though, just not super relevant for my day to day. I much prefer a faster, "weaker" model.
Fiduciary duty but for AI, interesting. I think there's some potential there, though of course you'll end up confronting the classic sci-fi trope of "what if the system judges what's best for the user in a way that is unexpected / harmful"? But, solve that with strong guardrails and/or scoping and you might have something.
I'm starting to get to the point where I'll only listen to AI energy use critiques if the commentator tells me up front they abstain from all forms of social media, especially video-based social media, first.
Note that I did not criticise the AI energy. I criticised tech as a whole. Tech is part of the problem (the problem here being "we are killing our only planet").
Are they? Or do you just mean that it's few and far between that we hear about them? If it's the former, I think there's a much bigger universe of this kind of stuff than most people realize. Otoh, if you're just commenting on the lack of coverage, then, yeah I agree I wish more publicity was paid to small software like this. Maybe we need a catchy term - "organic software"? "Locally grown software"?
I talked to my friends who aren't in tech a lot about what they would want with software. A lot of the benefits of small software like this would actually be compliance and reporting issues with non-profit. Sifting through large amounts of data with very unstructured inputs.
The actual community building is fairly not as automated unless you have very specific problems. Like even in the example above, having an automated message is useful but staffing the team to handle when things are NOT in a good spot would probably be the real scaling cost.
We're in a transition phase, but this will shake out in the near future. In the non-professional space, poorly built vibecoded apps simply won't last, for any number of reasons. When it comes to professional devs, this is a problem that is solved by a combination of tooling, process, and management:
(1) Tooling to enable better evaluation of generated code and its adherence to conventions and norms
(2) Process to impose requirements on the creation/exposure of PRDs/prompts/traces
(3) Management to guide devs in the use of the above and to implement concrete rewards and consequences
Some organizations will be exposed as being deficient in some or all of these areas, and they will struggle. Better organizations will adapt.
The unfortunate reality is that (1) and (2) is what many, many engineers would like to do, but management is going EXACTLY in the opposite direction: go faster! Go faster! Why are you spending time on these things
I think this is an interesting point, my one area of disagreement is that there is no "anti-LLM sentiment" in the programming community. Sure, plenty of folks expressing skepticism or disagreement are doing so from a genuine place, but just in reading this site and a few email newsletters I get I can say that there is a non-trivial percent in the programming world who are adamantly opposed to LLMs/AI. When I see comments from people in that subset, it's quite clear that they aren't approaching it from a place of skepticism, where they could be convinced given appropriate evidence or experiences.
But there's a difference. Being opposed to AI-generated art/music/writing is valid because humans still contribute something extraordinarily meaningful when they do it themselves. There's no market for AI-generated music, and AI-generated art and writing tends to get called out right away when it's detected. People want the human expression in human-generated art, and the AI stuff is a weak placeholder at best.
For software the situation is different. Being opposed to LLM-generated software is just batshit crazy at this point. The value that LLMs provide to the process makes learning to use them, objectively, an absolute must; otherwise you are simply wasting time and money. Eric S. Raymond put it something like "If you call yourself a software engineer, you have no excuse not to be using these tools. Get your thumb out of your ass and learn."
Ok, I’ll bite. What’s there to learn that you can tie directly to an increase of productivity?
I can say “learn how to use vim makeprg feature so that you can jump directly to errors reported by the build and tool” and it’s very clear where the ROI. But all the AI hypers are selling are hope, prayers, and rituals.
The skill is learning to supply the LLM with enough context to do anything a developer does: turn specs into code, check its work including generating and running tests, debug and analyze the code for faults or errors, and run these in a loop to converge on a solution. If you're about to do something by hand in an IDE, STOP. Think about what the LLM will need to know to perform that task for you.
It may take some human intervention, but the productivity results are pretty consistent: tasks that used to take weeks now take hours or days. This puts in reach the ability to try things you wouldn't countenance otherwise due to the effort and tedium involved. You'd have to be a damn fool not to take advantage of the added velocity. This is why what we do is called "engineering", not a handicraft.
I’m not an AI hyper, I just don’t code manually anymore. Tickets take about as much time to close as before, but the code shipped now has higher test coverage, higher performance, better concurrency error handling, less follow-up refactor PRs, less escapes to staging/prod and better documentation; some of it is now also modeled in a model checker.
And as you have written on your blog it's a soft cap that can be exceeded with justification.
reply