Google Trends make it seem like we're out of the exponential growth phase for LLMs-- search interest is possibly plateauing.
A decline in search interest outside of academia makes sense. The groups who can get by on APIs don't care so much how the sausage is made and just want to see prices come down. Interested parties have likely already found tools that work for them.
There's definitely some academic interest outside of CS in producing tools using LLMs. I know plenty of astro folks working to build domain specific tools with open models as their backbone. They're typically not interested in more operational work, I guess because they operate under the assumption that relevant optimizations will eventually make their way into public inference engines.
And CS interest in these models will probably sustain for at least 5-10 more years, even if performance plateaus, as work continues into how LLMs function.
All that to say, maybe we're just seeing the trend die for laypeople?
Well, Google Search trends are also only an imperfect proxy for what we are actually interested in.
Eg tap water is really, really useful and widely deployed. Approximately every household is a user, and that's unlikely to change. But I doubt you'll find much evidence of that in Google Search trends.
well gary marcus a non lay person is helping spread word that ai winter is again upon us.
but maybe statistical learning from pretraining is near its limit. not enough data or not enough juice to squeeze more performance out of averages.
though with all the narrow ais it does seem plausible you might be able to cram all what these narrow ais can do in on big goliath model. wonder if reinforcement learning and reasoning can manage to keep the exponential curve of ai going if there are still hiccups in the short term.
the difficulty in just shoehorning llms as they are in any and every day task without a hitch might be behind the temporary hype-dying down trend.
But "Large language model" as a topic in google trends is still in its peak. Maybe just everyone who would be the audience is already knowledgeable about LLMs so why would Google Search trends be able to keep rising?
ChatGPT is at it's peak, and something like Claude is still rising.
A decline in search interest outside of academia makes sense. The groups who can get by on APIs don't care so much how the sausage is made and just want to see prices come down. Interested parties have likely already found tools that work for them.
There's definitely some academic interest outside of CS in producing tools using LLMs. I know plenty of astro folks working to build domain specific tools with open models as their backbone. They're typically not interested in more operational work, I guess because they operate under the assumption that relevant optimizations will eventually make their way into public inference engines.
And CS interest in these models will probably sustain for at least 5-10 more years, even if performance plateaus, as work continues into how LLMs function.
All that to say, maybe we're just seeing the trend die for laypeople?