There's only one problem with a race to superintelligence, and that's that nobody has evidence that mere intelligence is coming, much less superintelligence.
(There are a thousand more problems, but none of them matter until that first one is overcome.)
Today's AI does exhibit some intelligence, period. It is absurd to claim that an intelligent-looking entity doesn't have intelligence, only because we might not be able to determine which part of the entity has one. The superintelligence is an entirely different problem though because there is no clear path from intelligence to so-called superintelligence, everything has been just a speculation so far.
> It is absurd to claim that an intelligent-looking entity doesn't have intelligence
Is it? I am pretty sure biology will solve good old "are viruses alive?" sooner than we agree on definition of intelligence. "Chinese Room" is at least 40 years old.
And so do tons of counterarguments against the Chinese Room argument.
Practically speaking, the inherentness of intelligence doesn't really matter because both intelligent-looking entity and provably intelligent entity are capable for societal disruptions anyway. I partly dislike the Chinese Room argument for this reason; it facilitates useless discussions in most cases.
In that case there was still some intelligence. It turns out that a composite entity of Hans and its trainer was intelligent, and people (including the trainer) unknowingly regarded that as Hans' own intelligence.
Good gods, I can't wait for a second AI winter. Maybe we'll come up with fundamental breakthroughs in a couple of decades and give it another college try?
For the folks who lived though it; were the Expert Systems boosters as insufferable in the 80s as the LLM people are now about the path to machine intelligence?
No, because they mostly got military funding, not private equity.
ARPA would throw relatively large sums of money at you, but demand progress reports and a testable goal. Very little got rolled out based on hype. (Let's not talk about vehicle design.) If your project didn't show signs of working, or not enough signs of working, funding ended.
Anything which met goals and worked, we now think of as "automation" or "signal recognition" or "solvers", not "intelligent systems".
(There are a thousand more problems, but none of them matter until that first one is overcome.)