Large language models (LLMs) demonstrate impressive capabilities across numerous domains, from programming and technical support to cooking instructions and research enhancement. However, they continue to struggle with precise symbol manipulation in tasks requiring rigorous logical reasoning, mathematical proofs, and complex algebraic operations. As a student of philosophy and computer science, I find this intersection especially fascinating because of the connections between logical precision and conceptual clarity.
Bell labs and Xerox PARC did great impactful work long after their parent companies were relevant and still do .
The fact Google did the initial work on transformers but only OpenAI was able to productize is an indictment of their stagnation more than an achievement.
I agree that they recognized the fundamental problem but I have different speculation as to what the problem is: the problem must be it's technologically too boring and obviously not profitable to them.
It's culmination of decades of NLP researches, but it's also just East Asian predictive text percussively adapted for variable word length language. It must've been clear to whoever core people that it's fundamentally a sluggish demo or at most a cheap outright-sold commodity product.
IMO they would be right unless someone finds a reason that LLMs can't ever be self hosted in the way Google Search can't be. They must have just saw by the way it is that LLM is at best a regularization engine for that magic box.
2017, seven Google employees invent the transformer architecture and publish a paper. Google's investing heavily into ML, with their own custom 'TPU' chips and their 'Tensorflow' ML framework.
2019ish, Google has an internal chatbot they decide to do absolutely nothing with. Some idiot tells the press it's sentient, and they fire him.
2022, ChatGPT launches. It proves really powerful, a product loads of individuals and businesses are ready to pay for, and the value of the company skyrockets.
2023, none of the seven Transformer paper authors are at Google any more. Google rushes out Bard. Turns out they don't have a sentient super-intelligence after all. In fact it's badly received enough they end up needing to rebrand it a few months later.
Classic tortoise-and-hare situation - Google spent 5 years napping, then had to sprint flat out just to take third place.
Have you ever listened to what Lemoine said? Sure, we have no proof and he's under NDA so probably no documentation that can be scrutinized. But still, his alleged chats were chilling in some ways. They probably didn't except him to go public and so they had to spend years nerfing their chat bot before launching it as a product and that's why it sucks: They're too careful and have too much to lose in bonuses. Google will probably lose some market share over the next few years before they're getting nervous to put someone with a longer leash into the CEO seat.
I recall this particular person seeming like a bit of a crackpot on internal forms before (and for reasons unrelated to) the Lamda chatbot. I didn't know him personally and don't even remember the details anymore but it made an impression that wasn't dispelled by his reaction to a new model passing the turing test.
For the tortoise to win in technology it needs to be dedicated to relentlessly polishing and improving something over a long period to make the best product experience. Those aren't traits I particularly associate with Google unfortunately.
It is easier to judge revenue or market share than technical quality of the models itself objectively , they are relatively close to each other functionally .
In the market, I would say both Anthropic and openAI have been able to do that much better than traditional big tech including Google.
OpenAI is the market leader by far with the most name recognition. Google was the last to market. Its initial release of Gemini was a total flop because of the meme "Use elmer's glue on pizza to keep the cheese on". It has finally become more consistent, and it manages to compete with other models though I never see anyone recommending Gemini first.
All of these companies are in the red, but OpenAI has the most revenue.
This is a bit of a tangent, but I don't think OpenAI's brand is all that durable. You can see that Perplexity.AI has been gaining rapidly. At this point they have half as much search traffic as OpenAI:
It was used in Google translate, and BERT was incorporated into search in 2019, though I don't think it was a clear win for search, I feel like I started having to add exact quotes to everything technical/programming around then.
One thing I don't understand is google has so much metadata on search sessions to RLHF their search results.
E.g. when I start a search session to solve a programming problem (before llms), I will continually search different terms to get to my solution webpage. Then stop. This session metadata and the path I took is highly significant data that can be used to help llms recognise what research itself looks like.
Not RLHF, but my understanding was they heavily use that data and it was a big part of their moat, part of why competitors wanted to clone their results because they couldn't derive as good of quality from the web alone (Microsoft used the bing toolbar to clone them in the 2010s).
Where did you come up with this ? This is just not true, that Pythagoras had little interest in math. He had a love of numbers and thought that math was a way to the divine or at least understanding the divine. His philosophy, not religion , but philosophy was a way of life that entangled mathematics profusely.
Synapticyte is my startup dedicated to stopping school and workplace shootings using advanced AI technology. We’ve developed a highly accurate model for identifying firearms and are working on building a robust alert system. This system will enable schools or organizations to notify police, faculty, and staff when weapons are detected. Our model leverages a combination of Convolutional Neural Networks (CNN) and transformers. Although it’s still in the early stages, we believe this is a powerful and important application of AI technology.
Synapticyte is my startup dedicated to stopping school and workplace shootings using advanced AI technology. We’ve developed a highly accurate model for identifying firearms and are working on building a robust alert system. This system will enable schools or organizations to notify police, faculty, and staff when weapons are detected. Our model leverages a combination of Convolutional Neural Networks (CNN) and transformers. Although it’s still in the early stages, we believe this is a powerful and important application of AI technology.
MIND is a simple, educational framework for building feedforward neural networks. Includes tensor operations, activation functions, and backpropagation. Ideal for learning core deep learning concepts and exploring Racket. Future updates will be released!
Introducing BlokAv it’s a modern antivirus utilizing blockchain to allow for rapid updates and community consensus on voting and recognizing malicious files. This is still a work in progress. I did this to learn and also to give something to the community. There also aren’t many open source AV options besides ClamAV.
Behavior analysis, signature scanning, sandboxing for Linux, blockchain implementation etc are deployed but still being worked on. A windows update will come in the future. I screwed the code up and just decided to restart it. The signature scan for now requires a hardcore path to a hash list. I’m working on downloading the list from a server. I’d love any feedback etc.
