Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful.
Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema.
One problem that is not resolved yet is that it still hallucinates a lot of attributes. For example we have tool that allows it to create contacts in user's CRM. I ask it to:
ChatGPT can be usefully for many things, but you should really, not use it if you want to retrieve factual data. This might partly be resolved by querying the internet like bing does but purely on the language model side these hallucinations are just an unavoidable part of it.
I would never rely on an LLM as a source of such information, just as I wouldn't trust the general knowledge of a human being used as a database. Does your workflow include a step for information search? With the new json features, it should be easy to instruct it to perform a search or directly feed it the right pages to parse.
Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema.
One problem that is not resolved yet is that it still hallucinates a lot of attributes. For example we have tool that allows it to create contacts in user's CRM. I ask it to:
"Create contacts for top 3 Barcelona players:.
It creates an structure like this"
1. Lionel Messi - Email: [email protected] - Phone Number: +1234567890 - Tags: Player, Barcelona
2. Gerard Pique - Email: [email protected] - Phone Number: +1234567891 - Tags: Player, Barcelona
3. Marc-Andre ter Stegen - Email: [email protected] - Phone Number: +1234567892 - Tags: Player, Barcelona
And you can see it hallucinated email addresses and phone numbers.