There's not enough hours in the day for everyone to do everything.
> There is a whole modern line of thinking that leaders should be providing the context and skills to give high performing teams MORE agency over their work streams.
Yes, this is great for agency over implementation, because leaders do not have context to decide and dictate the What/How of implementing every single change or solution to a problem. And the implementers need to know the context to ensure they make decisions consistent with that context.
But "leaders providing the context" is very different from "everyone researching the context on their own." So where are leaders getting this context from? A not-very-differentiated pile of 1000 generalist engineers-who-also-talk-to-customers-frequently-and-manage-their-own-work-streams? Or do they build a team with specialists to avoid needing the majority of people to constantly context-switch in a quest to be all of context-gatherers, work-prioritizers, market-researchers, and implementation-builders?
There are many leaders that use information as a tool that serves their own needs.
They may have the context, but they are either too focused on their own job to share it, or actively manage dissemination so they can manipulate the organization.
In my experience, this is the typical operating mode, though I do not think it is sinister or malicious - just natural.
If you have M customer complaints, and each one risks a differently-sized N customers... you better try to triage that vs just playing whack-a-mole with whatever comes to a random engineer first. I've never seen engineers plow through a bunch of 0-or-1-customers-would-actually-churn-over-this papercuts because it was easy and it feels good - the customer mentioned it! i fixed it! - while ignoring larger showstoppers that are major customer acquisition and retention barriers.
Nothing is knowable in only the same way that plans are useless but planning is essential.
There are very good less-cynical reasons. I've also seen companies with the opposite problem, where the engineers constantly shoot down real, important feedback brought by customer support in order to preserve the superiority of engineering over support.
If you have ten engineers and even just 100 customers, you have a very high number of conversational edges. Good luck keeping things consistent and doing any sort of long-term planning if engineers are turning the output of those conversations directly into features. "Engineers talking to customers but not making any changes" would be more stable, but is still a very expensive/chaotic way to gather customer feedback.
Additionally, very few of those single engineers have a full knowledge of the roadmap and/or the ability to unilaterally decide direction based on some of the customer feedback or questions. "Will this get fixed in the next two weeks?" "Will you build X?" etc. You don't want your customers getting a bunch of inconsistent broken promises or wrong information.
The best-managed orgs I've seen have pretty heavy engineering and user experience in their product and support orgs. You need people in those roles with knowledge of both how it's built AND how it should be used, but you can't continually cram all that knowledge into every single engineer.
A startup should start with the builders talking directly to the customers. But at a some point, if successful, you're going to have too many people to talk to and need to add some intermediaries to prevent all your engineering time going to random interrupts, and centralization of planning responsibilities to ensure someone's figuring out what's actually the most important feedback, and that people are going to work on it.
On the contrary, the best products are typically built by the users of the products. If you are building a product you don't use, it will be worse than if you used it.
Users should be everywhere, in and out of engineering.
> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.
for simple stuff, sure, React was ALWAYS inefficient. Even Javascript/client-side logic is still overkill a lot of the times except for that pesky "user expectations" thing.
for anything codebase that's long-lived and complex, combinatorics tells us how it'll near-impossible to have good+fast test coverage on all that.
part of the reason people don't roll their own is because being able to assume that the library won't have major bugs leads to an incredible reduction in necessary test service, and generally people have found it a safe-enough assumption.
throwing that out and trying to just cover the necessary stuff instead - because you're also throwing out your ability to quickly recognize risky changes since you aren't familiar with all the code - has a high chance of painting you into messy corners.
"just hire a thousand low-skilled people and force them to write tests" had more problems as a hiring plan then just "people are expensive."
Nobody wants to wait for those cycles to happen in the sorts of businesses that feature most prominently on HN. That flow works much better for "take existing business, with well defined flows, computerize it" than "people would probably get utility out of doing something like X,Y,Z, let's test some crap out."
Now, later-stage in those companies, yes, part of the reason for the chaos is because nobody knows or cares to reconcile the big-picture, but there won't be economic pressure on that without major scaling-back of growth expectations. Which is arguably happening in some sectors now, though the AI wave is making other sectors even more frothy than ever at the same time in the "just try shit fast!" direction.
But while growth expectations are high, design-by-throwing-darts like "let's write a bunch of code to make it easy to AB test random changes that we have no theory about to try to gain a few percent" will often dominate the "careful planning" approach.
> Nobody wants to wait for those cycles to happen in the sorts of businesses that feature most prominently on HN.
Bryce's Law: "We don't have enough time to do things right. Translation: We have plenty of time to do things wrong." Which was definitely true for YC startups, FAANGs, and the like in the ZIRP era, not so much now.
Systems development is a science, not an art. You can repeatably produce good systems by applying a proven, tested methodology. That methodology has existed since 1971 and it's called PRIDE.
> That flow works much better for "take existing business, with well defined flows, computerize it" than "people would probably get utility out of doing something like X,Y,Z, let's test some crap out."
The flows are the system. Systems development is no more concerned with computers or software than surgery is with scalpels. They are tools used to do a job. And PRIDE is suited to developing new systems as well as upgrading existing ones. The "let's test some crap out" method is exactly what PRIDE was developed to replace! As Milt Bryce put it: "do a superficial feasibility study, do some quick and dirty systems design, spend a lot of time in programming, install prematurely so you can irritate the users sooner, and then keep working on it till you get something accomplished." (https://www.youtube.com/watch?app=desktop&v=SoidPevZ7zs&t=47...) He also proved that PRIDE is more cost-effective!
The thing is, all Milt Bryce really did was apply some common sense and proven principles from the manufacturing world to systems development. The world settled upon mass production using interchangeable parts for a reason: it produces higher-quality goods cheaper. You would not fly in a plane with jet engines built in an ad-hoc fashion the way today's software is built. "We've got a wind tunnel, let's test some crap out and see what works, then once we have a functioning prototype, mount it on a plane that will fly hundreds of passengers." Why would a company trust an information system built in this way? It makes no sense. Jet engines are specced, designed, and built according to a rigorous repeatable procedure and so should our systems be. (https://www.modernanalyst.com/Resources/Articles/tabid/115/I...)
> Which is arguably happening in some sectors now, though the AI wave is making other sectors even more frothy than ever at the same time in the "just try shit fast!" direction.
I think the AI wave will make PRIDE more relevant, not less. Programmers who do not upskill into more of a systems analyst direction will find themselves out of a job. Remember, if you're building your systems correctly, programming is a mere translation step. It transforms human-readable specifications and requirements into instructions that can be executed by the computer. With LLMs, business managers and analysts will soon be able to express the inputs and outputs of a system or subsystem directly, in business language, and automatically get executable code! Who will need programmers then? Perhaps a very few, brilliant programmers will be necessary to develop new code that's outside the LLMs' purview, but most business systems can be assembled using common, standard tools and techniques.
Bryce's Law: "There are very few true artists in computer programming, most are just house painters."
The problem is, and always has been, that all of systems development has been gatekept by programmers for the past few decades. AI may be the thing that finally clears that logjam.
You say "My Claude Code Setup" but where is the actual setup there? I generally agree with everything about how LLMs should be called you say, but I don't see any concrete steps of changing Claude Code's settings in there? Where are the "35 agents. 68 skills. 234MB of context."? Is the implementation of the "Layer 4" programs intended to be left to the reader? That's hardly approachable.
I just don't have much faith in "if you're doing it right the results will be magically better than what you get otherwise" anymore. Any single person saying "the problems you run into with using LLMs will be solved if you do it my way" has to really wow me if they want me to put in effort on their tips. I generally agree with your why of why you set up like that. I'm skeptical that it will get over the hump of where I still run into issues.
When you've tried 10 ways of doing it but they all end up getting into a "feed the error back into the LLM and see what it suggests next" you aren't that motivated to put that much effort into trying out an 11th.
The current state of things is extremely useful for a lot of things already.
That's completely fair, I also don't have much faith in that anymore. Very often, the people who make those claims have the most basic implementation that barely is one.
I'm not sure if the problems you run into with using LLMs will be solved if you do it my way. My problems are solved doing it my way. If I heard more about your problems, I would have a specific answer to them.
These are the solutions to where I have run into issues.
For sure, but my solutions are not feed the error back into the LLM. My solutions are varied, but as the blog shows, they are move as much as possible into scripts, and deterministic solutions, and keep the LLM to the smallest possible scope.
The current state of things is extremely useful for a subset of things. That subset of things feels small to me. But it may be every thing a certain person wants to do exists in that subset of things.
It just depends. We're all doing radically different things, and trying very different things.
I certainly understand and appreciate your perspective.
My basic problem is: "first-run" LLM agent output frequently does one or more of the following: fails to compile/run, fails existing test coverage, or fails manual verification. The first two steps have been pretty well automated by agents: inspect output, try to fix, re-run. IME this works really well for things like Python, less-well for things like certain Rust edge cases around lifetimes and such, or goroutine coordination, which require a different sort of reasoning than "typical" procedural programming.
But let's assume that the agents get even better at figuring out the deal with the more specialized languages/features and are able to iterate w/o interaction to fix things.
If the first-pass output still has issues, I still have concerns. They aren't "I'm not going to use these tools" concerns, because I also sometimes write bugs, and they can write the vast majority of code faster than I can.
But they are "I'm not gonna vibe-code my day job" concerns because the existence of trivially-catchable issues suggests that there's likely harder-to-catch issues that will need manual review to make sure (a) test coverage is sufficient, (b) the mental model being implemented is correct, (c) the outside world is interacted with correctly. And I still find bugs in these areas that I have to fix manually.
This all adds up to "these tools save me 20-30% of my time" (the first-draft coding) vs "these agents save me 90% of my time."
So I'm kinda at a plateau for a few months where it'll be hard to convince me to try new things to try to close that 20-30% -> 90% number.
I experience the same things. What I’ve found is there is no issue I can’t solve so it doesn’t repeat.
The real issue is I don’t know the issues ahead of time. So each experience is an iteration stopping things I didn’t know would happen.
Thankfully, I’m not trying to sell anyone anything. I don’t even want people to use what I use. I only want people to understand the why of what I do, and how it adds me value.
I think it’s important to understand this thing we use as best we can.
The personal value you can get, is entirely up to your tolerance for it.
Damn, it really is all just vibes eh? Everyone just vibes their way to coding these days, no proof AI is actually doing anything for you. It's basically just how someone feels now: that's reality.
In some sense, computers and digital things have now just become a part of reality, blending in by force.
I mean, it’s not vibes. I make real projects, and the failures of AI doing it force me to make fixes so that it only ever fails doing that thing once. Then it no longer fails to do that thing.
But the things I am doing might not be the things you are doing.
If you want proof, I intend to release a game to the App Store and steam soon. At that point you can judge if it built a thing adequately.
No offense intended, I don't even know you at all, but I see people claim things like you did so often these days that I begin to question reality. These claims always have some big disclaimer, as yours does. I still don't know a single personal acquaintance who has claimed even a 2x improvement on general coding efficiency, not even 1.5x in general efficiency. Some of my coworkers say AI is good for this or that, but I literally just waste my time and money when I use it, I've never gotten good results or even adequate results to continue trying. I feel like I am taking crazy pills sometimes with all of the hype!
I hope you're just one of the ones who figured it out early and all the hype isn't fake bullshit. I'd much rather be proven wrong than for humanity to have wasted all this time and resources.
I think the correct approach is to be skeptical. You should push back.
I think of this stuff as trivial to understand from my point of view. I am trying to share that.
I have nothing to sell, I don’t want anyone to use my exact setup.
I just want to communicate the value as I see it, and be understood.
The vast majority of it all is complete bullshit, so of course I am not offended that I may sound like 1000 other people trying to get you to download my awesome Claude Code Plugins repo.
(I'm not really sure LLMs will make it that much worse here, but all those things have been harmful to workers already.)
reply