It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'.

I would say that, except for the observable and testable performance, what else can you say about understanding?

It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python.

The mechanistic way this understanding arises is different than humans.

How can you say then it is 'not real', without invoking the hard problem of consciousness, at which point, we've hit a completely open question.

I think it is fair to say that AIs do not yet "understand" what they say or what we ask them.

When I ask it to use a specific MCP to complete a certain task, and it proceeds to not use that MCP, this indicates a clear lack of understanding.

You might say that the fault was mine, that I didn't setup or initialize the MCP tool properly, but wouldn't an understanding AI recognize that it didn't have access to the MCP and tell me that it cannot satisfy my request, rather than blindly carrying on without it?

LLMs consistently prove that they lack the ability to evaluate statements for truth. They lack, as well, an awareness of their unknowing, because they are not trying to understand; their job is to generate (to hallucinate).

It astonishes me that people can be so blind to this weakness of the tool. And when we raise concerns, people always say

"How can you define what 'thinking' is?" "How can you define 'understanding'?"

These philosophical questions are missing the point. When we say it doesn't "understand", we mean that it doesn't do what we ask. It isn't reliable. It isn't as useful to us as perhaps it has been to you.

To be fair, it can be hard to define “chair” to the satisfaction of an unsympathetic judge.

“Do chairs exist?”:

https://m.youtube.com/watch?v=fXW-QjBsruE

You can make categorical judgements on many things that are hard to define. Often by taking the opposite approach. It's hard to define what consciousness itself is yet I can define what it isn't and that teases a lot out.

So I can categorically say LLM's do not understand by quite literally understanding what NOT understanding is.

We know what LLM's are and what they are NOT.

Please see my earlier comment above:

> LLM's do not think, understand, reason, reflect, comprehend and they never shall.

How do you know what kind of "understanding" a python has? Why python and not a lizard? Or a bird? What method do you use for evaluating this? Does a typical python do what you tell your ai agent to do?...

C'mon, this comparison seems to be very, very unscientific. No offense...

I'm pretty sure parent poster meant they understand Python, the programming language, not that they have the understanding of the animal python.

As someone who was an engineer on the original Copilot team, yes I understand how tech works.

You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works.

Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on.

At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than SOTA LLMs.

I expect to see responses like yours on Reddit, not HN.

Before one may begin to understand something one must first be able to estimate the level of certainty. Our robot friends, while really helpful and polite, seem to be lacking in that department. They actually think the things we've written on the internet, in books, academic papers, court documents, newspapers, etc are actually true. Where the humans aren't omniscient it fills the blanks with nonsense.

> Where the humans aren't omniscient it fills the blanks with nonsense

As do most humans. People lie. People make things up to look smart. People fervently believe things that are easily disproved. Some people are willfully ignorant, anti-science, anti-education, etc.

The problem isn't the transformer architecture... it is the humans who advertise capabilities that are not there yet.

> As do most humans. People lie. People make things up to look smart.

The main difference is that people know when they lie and make things up to look smart, LLM doesn't.

There are human beings who believe absolutely insane and easily disprovable things. Even in the face of facts they continue to remain willfully ignorant.

Humans can convince themselves of almost anything. So I don’t understand your point.

Yes because working on co-pilot makes one well educated in philosophy of mind.

The standard, meaningless, HN appeal to authority. "I worked at Google, therefore I am an expert on both stringing a baroque lute and the finer points of Lao cooking. "

Gemini 3 gives a nice explanation if asked "can you explain how you don't really understand anything"

I’ve worked with many neuroscience researchers during my career. At a minimum, I’m extremely well-read on the subject of cognition.

I am not going to lie or hide my experience. The world is a fucked up place because we no longer respect “authority”. I helped build one of these systems; my opinion is as valid as yours.

Yours is the “standard meaningless” response that adds zero technical insight. Let’s talk about supportive tracing or optimization of KV values during pre-training and how those factors impact the apparent “understanding” of the resulting model.

My brain is able to determine when it doesn't know something.

That does seem to be a bit important for any "intelligent" system.

So, if we find a way to ensure that transformers balk at hallucinating will you then say that they “understand” what they’re saying?

Because that’s what your comment indicates.

> At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than SOTA LLMs.

Humans are remarkably consistent in their behavior in trained environments. That's why we trust humans to perform dangerous, precise and high stakes tasks. Humans have the meta-cognitive abilities to understand when their abilities are insufficient or when they need to reinforce their own understanding, to increase their resilience.

If you genuinely believe humans hallucinate more often, then I don't think you actually do understand how copilot works.

There is a qualitative difference between humans and LLM's 'hallucinating' (if we can even apply this terminology to humans which I contend is in-appropriate).

I'd add a simple thought experiment. A poor student doing a multi-choice exam paper and subsequently achieving a poor mark lets say 30% and now a child of 10 years attempting the same paper and achieving say 50%. Looked quantitively a perspective arises attributing understanding to a child who has chanced 50% on a multi-choice paper when compared to a student having studied the subject at hand.

Qualitatively however and we know this intuitively it is certainly NOT the case that the child of 10 comprehends or understands more than the poor student.

Your response is crazy to me. Humans are known to be remarkably inconsistent in behavior. Please read ‘Thinking Fast and Slow’ or at least go back your HS psych 101 notes.

> Humans have the meta-cognitive abilities to understand when their abilities are insufficient or when they need to reinforce their own understanding

Two word reply: dunning kruger

> I expect to see responses like yours on Reddit, not HN.

I suppose that says something about both of us.

We could use a little more kindness in discussion. I think the commenter has a very solid understanding on how computer works. The “understanding” is somewhat complex but I do agree with you that we are not there yet. I do think that the paradigm shift though is more about the fact that now we can interact with the computer in a new way.

The end effect certainly gives off "understanding" vibe. Even if method of achieving it is different. The commenter obviously didn't mean the way human brain understands

You understand how the brain works right? It's probability distributions mapped to sodium ion channels. The human understands nothing.

You understand how the brain works?

You're the one then. All those laggardly neurobiologists are still struggling.

I've heard that this human brain is rigged to find what it wants to find.

Thats how the brain works, not how the mind works. We understand the hardware, not the software.

Are we even sure we understand the hardware? My understanding is even that is contested, for example orchestrated objective reduction, holonomic brain theory or GVF theory.

“You understand how the brain works right? It’s neurons and electrical charges. The brain understands nothing.”

I’m always struck by how confidently people assert stuff like this, as if the fact that we can easily comprehend the low-level structure somehow invalidates the reality of the higher-level structures. As if we know concretely that the human mind is something other than emergent complexity arising from simpler mechanics.

I’m not necessarily saying these machines are “thinking”. I wish I could say for sure that they’re not, but that would be dishonest: I feel like they aren’t thinking, but I have no evidence to back that up, and I haven’t seen non-self-referential evidence from anyone else.

You understand how reality works right?

It's all just atoms clinging to each other.

Simple.

Heh.

Birds and planes operate using somewhat different mechanics, but they do both achieve flight.

Birds and planes are very similar other than the propulsion and landing gear, and construction materials. Maybe bird vs helicopter, or bird vs rocket.

> other than the propulsion and landing gear, and construction materials

"Apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, the fresh water system and public health, what have the Romans ever done for us?" Monty Python's Life of Brian.

P.S. It is relative, but quite a lot of differences IMHO.

Yes indeed, but they still use wings, fly in the air and so on.

Artificial neural networks have very little in common with real brains and have no structural or functional similarities besides "they process information, and they have things called neurons". They can perform some of the same tasks though, like how a quadcopter can perform some of the duties as a homing pigeon.

"I don't "understand" how LLMs "understand" anything."

Why does the LLM need to understand anything. What today's chatbots have achieved is a software engineering feat. They have taken a stateless token generation machine that has compressed the entire internet's vocabulary to predict the next token and have 'hacked' a whole state management machinery around it. End result is a product that just feels like another human conversing with you and remembering your last birthday.

Engineering will surely get better and while purists can argue that a new research perspective is needed, the current growth trajectory of chatbots, agents and code generation tools will carry the torch forward for years to come.

If you ask me, this new AI winter will thaw in the atmosphere even before it settles on the ground.

Every time I see comments like these I think about this research from anthropic: https://www.anthropic.com/research/mapping-mind-language-mod...

LLMs activate similar neurons for similar concepts not only across languages, but also across input types. I’d like to know if you’d consider that as a good representation of “understanding” and if not, how would you define it?

Anthropic is pretty notorious for peddling hype. This is a marketing article - it has not undergone peer-review and should not be mistaken for scientific research.

it has a proper paper attached right at the beginning of the article

It’s not peer-reviewed, and was never accepted by a scientific journal. It’s a marketing paper masquerading as science.

If i could understand what the brain scans actually meant, I would consider it a good representation. I don't think we know yet what they mean. I saw some headline the other day about a person with "low brain activity" and said person was in complete denial about it, I would be too.

As I said then, and probably echoing what other commenters are saying - what do you mean by understanding when you say computers understand nothing? do humans understand anything? if so, how?

Does a computer understand how hot or cold it is outside? Does it understand that your offspring might be cranky because they’re hungry? Or not hungry, just tired? Can it divine the difference?

Does a computer know if your boss is mad at you or if they had a fight with their spouse last night, or whatever other reason they may be grumpy?

Can a computer establish relationships… with anything?

How about when a computer goes through puberty? Or menopause? Or a car accident? How do those things affect them?

Don’t bother responding, I think you get the point.

You don't understand how the tech works, then.

LLMs aren't as good as humans at understanding, but it's not just statistics. The stochastic parrot meme is wrong. The networks create symbolic representations in training, with huge multidimensional correlations between patterns in the data, whether its temporal or semantic. The models "understand" concepts like emotions, text, physics, arbitrary social rules and phenomena, and anything else present in the data and context in the same fundamental way that humans do it. We're just better, with representations a few orders of magnitude higher resolution, much wider redundancy, and multi-million node parallelism with asynchronous operation that silicon can't quite match yet.

In some cases, AI is superhuman, and uses better constructs than humans are capable of, in other cases, it uses hacks and shortcuts in representations, mimics where it falls short, and in some cases fails entirely, and has a suite of failure modes that aren't anywhere in the human taxonomy of operation.

LLMs and AI aren't identical to human cognition, but there's a hell of a lot of overlap, and the stochastic parrot "ItS jUsT sTaTiStIcS!11!!" meme should be regarded as an embarrassing opinion to hold.

"Thinking" models that cycle context and systems of problem solving also don't do it the same way humans think, but overlap in some of the important pieces of how we operate. We are many orders of magnitude beyond old ALICE bots and MEgaHAL markov chains - you'd need computers the size of solar systems to run a markov chain equivalent to the effective equivalent 40B LLM, let alone one of the frontier models, and those performance gains are objectively within the domain of "intelligence." We're pushing the theory and practice of AI and ML squarely into the domain of architectures and behaviors that qualify biological intelligence, and the state of the art models clearly demonstrate their capabilities accordingly.

For any definition of understanding you care to lay down, there's significant overlap between the way human brains do it and the way LLMs do it. LLMs are specifically designed to model constructs from data, and to model the systems that produce the data they're trained on, and the data they model comes from humans and human processes.

You appear to be a proper alchemist, but you can't support an argument of understanding if there is no definition of understanding that isn't circular. If you want to believe the friendly voice really understands you, we have a word for that, faith. The skeptic sees the interactions with a chatbot as a statistical game that shows how uninteresting (e.g. predictable) humans and our stupid language are. There are useful gimmicks coming out like natural language processing, for low risk applications, but this form of AI pseudoscience isn't going to survive, but it will take some time for research to catch up to understanding how to describe the falsehoods of contemporary AI toys

Understanding is the thing that happens when your neurons coalesce into a network of signaling and processing such that it empowers successful prediction of what happens next. This powers things like extrapolation, filling in missing parts of perceived patterns, temporal projection, and modeling hidden variables.

Understanding is the construction of a valid model. In biological brains, it's a vast parallelized network columns and neuron clusters in coordinated asynchronous operation, orchestrated to ingest millions of data points both internal and external, which result in a complex and sophisticated construct comprising the entirety of our subjective experience.

LLMs don't have the subjective experience module, explicitly. They're able to emulate the bits that are relevant to being good at predicting things, so it's possible that every individual token inference process produces a novel "flash" of subjective experience, but absent the explicit construct and a persistent and coherent self construct, it's not mapping the understanding to the larger context of its understanding of its self in the same way humans do it. The only place where the algorithmic qualities needed for subjective experience reside in LLMs is the test-time process slice, and because the weights themselves are unchanged in relation to any novel understanding which arises, there's no imprint left behind by the sensory stream (text, image, audio, etc.) Absent the imprint mechanism, there's no possibility to perpetuate the construct we think of as conscious experience, so for LLMs, there can never be more than individual flashes of subjectivity, and those would be limited to very low resolution correlations a degree or more of separation away from the direct experience of any sensory inputs, whereas in humans the streams are tightly coupled to processing, update in real-time, and persist through the lifetime of the mind.

The pieces being modeled are the ones that are useful. The utility of consciousness has been underexplored; it's possible that it might be useful in coordination and orchestration of the bits and pieces of "minds" that are needed to operate intelligently over arbitrarily long horizon planning, abstract generalization out of distribution, intuitive leaps between domains that only relate across multiple degrees of separation between abstract principles, and so on. It could be that consciousness will arise as an epiphenomenological outcome from the successful linking together of systems that solve the problems LLMs currently face, and the things which overcome the jagged capabilities differential are the things that make persons out of human minds.

It might also be possible to orchestrate and coordinate those capabilities without bringing a new mind along for the ride, which would be ideal. It's probably very important that we figure out what the case is, and not carelessly summon a tortured soul into existence.

Looks like somebody used AI to generate this response. XD

I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.

It could very well be that statistics and tokens is how our brains work at the computational level too. Just that our algorithms have slightly better heuristics due to all those millennia of A/B testing of our ancestors.

Except we know for a fact that the brain doesn’t work that way. You’re ignoring the entire history of neuroscience.

I didn't know that neuroscience even claimed to a full enough understanding of how people think to conclusively disprove this.

Certainly not a full understanding, but there’s enough understanding to refute vague claims that we are somehow analogous to a text model.

What do you mean by “understand”? Do you mean conscious?

Understand just means “parse language” and is highly subjective. If I talk to someone African in Chinese they do not understand me but they are still conscious.

If I talk to an LLM in Chinese it will understand me but that doesn’t mean it is conscious.

If I talk about physics to a kindergartner they will not understand but that doesn’t mean they don’t understand anything.

Do you see where I am going?