What if I told you there are no safety guardrails. I used GLM 5.1 and had fable literally build a harness to avoid triggering guard rails.
I built skills carefully and had Fable doing vuln research and exploit repro in a few hours. I called the project manhattan. The GLM models are down for almost anything so I named it Oppenheimer. It orchestrated the fable CLI agents via tmux. This whole Fable/Mythos thing is such a fucking joke. It is all PR and theatre and they know it.
I’ve been doing pentesting with LLMs for a while and only hit a few “nope I won’t do that” and one “this conversation is flagged for being against the TOS”. No idea what the guardrails are but they are trivially abused
I don't even care. It is the same problem advent of code had as a public challenge with a leader board. I now mostly just think either embrace the LLM or keep it to a more in person or vetted audience. But, again, if you create a competition in the spirit of humans without LLMs and that is in the rules and someone uses an LLM that is on them IMO. I am sad advent of code decided to end their competition. LLMs are here to stay, let's embrace that and see what the new universe of competitions with LLMs can be. There will always be a place for human only competition, but for public facing ones LLM accepted is the only tenable position.
This does bring "Pay to compete" concerns and create incentive structures that encourage more LLM use. I don't know what to do about it.
Anthropic has a vested interest in downplaying the harness relevance. In my experience harness really matters. More capable models are great, but current models are enough if you put some engineering effort into the harness.
You have to do what I call "Manhattan Project" them. You can almost always evade the controls by carefully prompting them. It just wastes effort and time you should be spending doing other things in an LLM workflow. Essentially, there is almost no single discrete piece of a reverse engineering or CTF process that you can't get Claude to do, you just have to isolate it adequately and avoid letting it use names that attenuate it towards "this is an exploit" or "this is reverse engineering". I have not found a task I could not convince Claude to do. You can also fill the context window up with badgering it and eventually it is likely to simply let you through if you are careful, most of the safe guards are not deterministic.
Anthropic made their models very averse to reverse engineering and vulnerability research chores. It is a difficult problem, but attackers will use models like GLM and defenders will be stuck with security engineering averse models.
Google is also a Cloud Provider. Cloud is now ~18% of Google. While it is an advertising juggernaut. Cloud is also rapidly growing, so the local models simply fit as AI research and dev and getting more people on Gemini models. They /are/ advertising, effectively :)
Don't LLMs work on attention though? The closer in their hyperdimensional space you can land your problem to their inherent understand the better they are at understanding your problem domain. RAG loops can be very slow and agents may simply lack the knowledge to use them correctly.
But, in short, the ability to manage information, to process it properly, is more important in this regard than just having the information. "Having" more knowledge is not a guarantee to "using" it better.
And to improve reliability, if the machine can check, it will have to check. "Costly" cannot be an excuse.
Your suggestion is not clear: yes we reason and define relevant details (maybe through further information retrieval) to better construct queries - that is what Analytical school of thought taught and insisted on -, and even more crucial is that the subsequent delegated steps, of constructing replies, imply reasoning and information retrieval.
Said abilities - intellectual strength - are immensely more important than notions. The relation between network size and intellectual strength, vs network size and notions (original topic in this branch), is presumably not yet that clear. Intelligent models may not necessarily be embedded with explicit information of everything, though they will have to have ways to reach that upon contingent necessity (to solve specific problems). Like us.
I agree with what you said. I just wanted to add that intelligent models probably need to have some notion embedded (but not everything), as some information retrieval is not trivial. Too few embedded notions will hurt it's ability to solve problems but from some point onward you'll get diminishing returns (where it starts to make sense to rely just on information retrieval).
For example, you if you instruct a model to create decoder for some data type users will upload to your website. The intelligent model without notions will retrieve information about that data type and build a working decoder, but it might miss from context that users uploading to a website means untrusted input and thus won't even try to gather information about what it needs to be done to securely handle such uploaded data.
Or if you give it a task to translate text to a language it didn't encounter during training. You can provide it with grammar rules and a dictionary for information retrieval, but I guess it won't perform as well as inteligent model that already has some fundamental notions of that language and only needs a dictionary to expand its vocabulary.
Gpt-4.1 only knows a lot of patterns, but doesn't have reasoning intelligence that would help it properly use that knowledge. So, a small reasoning model can easily beat it in a lot of tasks. The question is how will, 14 months from now, new small reasoning models compare to current big reasoning models.
How much information needs to be embedded is not yet clear, but currently, bigger reasoning models are still better at complex tasks than small reasoning models. Either sweet spot of embedded notions is higher that what current small models have or information retrieval ability needs to improve.
This assumes that every designer is on the bell curve at the big tech firms in the roles that can influence this. I am not defending modern UI/UX, but that is quite an assumption.
You're right, it is a big assumption, but it's not unfounded. I've worked at F50 companies and founded my own startup. It's a lot easier to get two really great designers from the far right side of the bell curve when you're a small startup. As an organization reaches even 1000 people, you're now starting to draw from the middle of the bell curve. In fact, you have to. If you try to hire only from the far right of the bell curve for all positions, you end up with a lot of egos that will clash. In the best case, you hire leaders from the far right of the bell curve and followers from the middle. But at some point those mediocre followers start asking for promotions and your hotshot leader leaves for greener pastures. Controlling for that in your hiring practices at a large organization is virtually impossible, particularly if you have standard (middle of the bell curve) HR people. BTW, this is exactly why startups out-execute large companies every day of the week. A small startup can carefully control its hiring, select from the right side of the bell curve, and avoid all the large company HR crap. But as soon as it starts to scale, the bell curve becomes a looming threat.
On the other hand, just working at big tech doesn't mean you are especially great. Conformance and criteria other than raw skill matter. As you say, promotion games, etc... I would just lump all of that under conformance. So, you aren't wrong.
However, why startups outperform big companies isn't just the skill gap. Even if you have the most amazing leadership in big tech it is monumentally difficult to move the needle on some problems purely because of size not because of incompetence. All I am saying is don't overindex on perceived intelligence. A big org can start looking pretty dumb even though it is still far right of the bell curve compared to even a startup (hypothetically). Org size and the constraints that brings are a significant factor.
Oh, absolutely size has a lot to do with it, too. A founder and three engineers can change the plan at lunch to respond to something they see in the market. It takes admin assistants days, possibly a week or two to get the meetings set up and arrange the catered lunch for the same decision to be made at a large company. And even when it’s made, half the company will spend months trying to undermine it.
reply