This is essentially an open research question. ML theory is unfortunately very weak relative to where the empirics are. I think there's a relatively optimistic paper that was posted a while back here but I would also take it with a grain of salt.
There's of course empirical results and relatively weak theoretical results like the UAT but I also don't think that answers your question fully, especially since it seems impossible to definitively answer questions that the industry seems to betting on like whether or not there is a lower bound to their error rate or whether hallucination as a problem can be solved. We have much stronger ideas of what linear regression is doing relative to what LLMs are doing.
considering they work with any architecture/configuration given enough compute, just more or less efficiently - then maybe it's fundamental, in the same sense as why electricity works...
Why does linear regression works? Why does computer works? Because it's about math and the encoding information. If we can encode words as numbers, then why can't we encode their order as a relation? It's just that neural networks are very apt at finding that relation even if it's noisy.
Yes of course I don't actually hate what school is now. Not directly. How could I, I'm not even allowed to observe it! But I definitely hated what I had to do and it did not work for me. And that is useful information when I'm helping my kids.
Was this true when you were a kid? Why do you think it changed? Because when I was a kid and a kid was bad, the teacher would make the parent come to class until the kid started behaving. Do you think this would work today? And why would some teachers be opposed to it?
Indeed. Children and not "little adults". They are emotionally and intellectually immature, literally with the brain and body growing into to the capabilities of an adult.
And if good habits are not instilled, they will have a difficult life ahead of them. It's far easier to learn those habits when young, than to try to independently course correct as an adult.
Not coercing a child towards correct behaviours, is doing them a great disservice. In some circumstances, it's child abuse to not coerce those bahaviours.
There's a huge difference between a loving parent gently but firmly teaching their kid to clean their teeth every day even though they don't want to, and a brutal schoolteacher beating facts into a class full of miserable kids.
There is a difference. Yet that's the point. Because "beating facts" and "miserable kids" is not necessarily inferred from 'coerced', as opposed to 'gently but firmly', and that's why people are clarifying so.
There is in fact little difference between "you have to learn these things" and "brushing your teeth".
I don't either - I'm am anarchist. But, ever hear the saying, "against all authority except mommy?" Kids need some level of coercion just to keep them alive. They have to be made to even eat sometimes.
Not the poster to whom you are questioning, but I would argue that inspiring and encouraging are much better than coercing, especially if the goal is to educate, as I am skeptical that coercion is ever going to work to get true learning.
In a way, I think coercion is a requirement to be ethical. Ethics is determined based on what current society believes to be the right thing to do. We see that there are a variety of different cultures and ethics around the world, which would indicate that humans wouldn't just automatically follow a universal set of rules.
Thus to be ethical in your society, usually means you must follow the rules determined by a collective group of your nations ancestors or you will be shunned/jailed/harmed/etc. Which is essentially coercion. "Act this way or be punished."
But there is a difference in behaving ethically and behaving legally. While there may be consequences for behaving unethically (IE "I won't do business with them because I do not feel they are ethical"), society generally only overtly punishes those who do things that are illegal.
White collar crime might be illegal but most societies would definitely punish a murderer either legally or illegaly. Social stigma is a MUCH more serious thing than legality of action.
reply