I was thinking along the same lines and I think where I end up is realizing searching though the possible space of token sequences isn't the way to do it as the text output space is too far removed from the ground truth for humans. As in text is already a biased projection of reality by humans, now we're searching through an LLM's crude estimation of a biased projection of this?
I think a deeper architectural change involving searching internally in the latent space of a model is required. That way we could search in "thought space" instead of "text space" and maybe then only convert the output to text after searching.
I do similar stuff, I'm just willing to learn a lot more at the cost of a small percent of my knowledge being incorrect from hallucinations, just a personal opinion. Sure human produced sources of info is gonna be more accurate (more not 100% still), and I'll default to that for important stuff.
But the difference is I actually want to and do use this interface more.
llama3 on groq hits the sweet spot of being so fast that I now avoid going back to waiting on gpt4 unless I really need it, and being smart enough that for 95% of the cases I won't need to.
async function tryit(col, value) {
col.valueAsNumber = value;
col.dispatchEvent(new Event('change'));
submit.click();
await new Promise(resolve => setTimeout(resolve, speed));
var res_text = result.innerText.split(/[ ()%]/)[4];
if (res_text === "Splendid!") {
throw new Error("Finished - Found correct combination");
}
col.score = parseInt(res_text);
return col.score;
}
async function binarySearch(col) {
let start = 0;
let end = 15;
let mid = 7;
let startAccuracy = await tryit(col, start);
let endAccuracy = await tryit(col, end);
let midAccuracy = 0;