It's usually not the same pile of meat defining the problem and solving the problem.
Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains.
The style of argument you made about the nature of the machinery used applies just as well (maybe better) to humans. To get a valid argument, we'll need to be more nuanced.
>> Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved.
> It's usually not the same pile of meat defining the problem and solving the problem.
True, but this distinction is also irrelevant.
The point is that problems capable of being solved by software systems are identified, reified, and then determined to be solved by people. Regardless of the tooling used to do so and the number of people involved.
> Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains.
But machines do not, and cannot, exceed humans in the domain of "understanding what a human wants" because this type of understanding is intrinsic to people by definition. Machines can do a lot of things, things which can be amazing and are truly beneficial to mankind, but they cannot understand as people colloquially use this term since they are not people.
I believe a decent analogy for this situation is how people will never completely understand the communication whales use with each other the way whales do themselves. There may someday exist the ability to translate their communication into a semblance of human language, but that would be only what we think is correct and not the same as being a whale.
What if systems develop that are able to understand typical human intention and wants with an accuracy that is superior to most humans?
You seem to rule this out, but despite having similar biology and wants, humans misunderstand others' intents and miss cues a lot.
--
Is it impossible that humans could build a system to know what a whale wants, based on its vocalization, that does better than the typical whale? Do we know that whales do really great at this, even?
> Is it impossible that humans could build a system to know what a whale wants, based on its vocalization, that does better than the typical whale?
The answer resides in the question. For a human to know what a whale wants is to have an understanding of what it is to be a whale, not deriving a Rosetta Stone[0] such that bidirectional vocalizations could be exchanged.
So is it possible to communicate with a whale? Sure.
But the infeasiblity of cross-species understanding is far more difficult than even what you identified above:
... despite having similar biology and wants, humans
misunderstand others' intents and miss cues a lot.
Now add to all of this the fact that computer algorithms are just that, and not even having the shared commonality of being mammals, and the possibility of a machine understanding what a human wants is fantastical at best.
I think this is a silly take. Not all species had evolutionary pressure for cross-organism understanding. A different species that has selective pressure for social understanding and greater mental understanding is probably going to understand what an insect wants better than say, another insect.
That is, how well you can understand someone else doesn't depend just on common ground, but perhaps even more on how mature of machinery for understanding that you have.
> That is, how well you can understand someone else doesn't depend just on common ground, but perhaps even more on how mature of machinery for understanding that you have.
I disagree, and believe I can explain why.
If I wrote:
I can think of a few HN posts I have written which did not
properly express my intent when I wrote them.
Is this a statement that you can relate with?
Even if not, does it communicate an experience you could envision based on your being a participant in this forum, having written posts yourself, and knowing I am a person just as you are doing the same?
What if I texted the sentence to someone who has never heard of HN?
Now what would happen if I wrote that same sentence on a piece of paper and handed it to a random person in a train station where no one spoke English?
Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains.
The style of argument you made about the nature of the machinery used applies just as well (maybe better) to humans. To get a valid argument, we'll need to be more nuanced.