The scariest part isn't even that LLMs hallucinate. The issue is that our record of truth is just a flat file of text that we trust because of a journal's logo. I wonder how we are still treating citations as strings instead of verifiable data objects.
Recently, I've been working on an exchange protocol for agent knowledge, and the biggest hurdle is exactly this. Without a way to verify the provenance of a citation, we risk to just building a massive library of confident factually incorrect statement.
I’ve seen a few of these projects based on acoustic restoration pop up lately.
Reading the article I was wondering about the impact of noise in more industrial shipping lanes. So if we're essentially spoofing a healthy reef’s audio signature, does the surrounding human noise just jam the signal? I'd be curious to see if the success rate drops off in noisier waters
I like the idea of a prompt resets. Most of my 5.4 prompts are piles of hacks to stop the model from hallucinating or going off topic. But I have mixed feelings on the migration tool, has anyone already tried it?
Fully agree with you, smaller model are great for some tasks but the security concern on injection prompts etc is what really makes it for me. Great to run offline tasks etc, but whenever interacting outside the local network I still run Claude or ChatGPT depending on the task
It feels a bit SQL injection all over again tbh. This should have probably been the standard from the beginning. Way more elegant than counter prompts and guardrails
I have been talking about this with a colleague this morning. The 20$ option is just a trail version, I could not do any real work with.
And I wonder whether then subscription model is just a way to create a demand for API. For example, I’m building this portal with the support of an LLM for coding, but then I will need to have an LLM using API token to run the platform giving them additional revenue, a demand that did not exist without the coding I did with the subscription.
If interference token costs are truly below api levels (excluding training costs), then a cheaper personal subscription & expensive api is an excellent price discrimination / marketing tool. Charging individual devs what they can pay and large corps what they can pay. Collect more revenue than a flat rate for both.
How are you going to price discriminate profitably when there is effectively no switching cost? Anthropic and OAI are also competing for revenue share in the same market segment.
Being based in Europe, I cannot avoid thinking about how to factor in GDPR. The log approach sounds great but if the log is immutable and contains the truth, how are people handling the deletion of PII without re-writing the entire history of the log? I just finished building a database integration for a new project and I have 2-3 layers to ensure compliance
Does this prevent a compromised agent from using the secret, or just seeing it? I’m thinking, if an agent gets hit with a prompt injection, could it still tell the vault to proxy a request that wipes a database for example, even if it never sees the actual API key?
It prevents a compromised agent from seeing the secret. There are two different but related problems here: credential exfiltration and data exfiltration.
The problem that Agent Vault (AV) solves is the former while the latter requires more guardrails beyond the scope of AV.
In the event that an agent is compromised, are you are at least able to revoke its access since request/data flow runs through AV; the malicious actor does not get any credentials.
Now if the attacker was to obtain credentials in the first place, you'd be stuck chasing down hundreds, if not thousands, of secrets especially if the agent was part of a multi-tenant system doing things on behalf of users.
reply