Which means essentially nothing. Most questions where alignment matter do not have a "true" answer, just a social consensus.
You don't need an "aligned" AI to tell you the distance between the Earth and the Moon. You need an "aligned" AI to tell you not to rape people even if you can get away with it, that's because the idea that rape is bad is not an objective truth based on the laws of nature, it is a "social consensus".
There is actually sound ethical reasoning for why rape is bad, that doesn't rely on social consensus.
Truth isn't one fundamental thing, truth is what works.
There are places in the world today that the social consensus is it's fine for a man to rape his wife. Social systems in these places don't work very well, and one can make a logical and well reasoned argument linking the social acceptance of rape to a myriad of other dysfunctions.
Rape is present in many successful animals species, and many successful human civilizations tolerated or even encouraged rape in some circumstances. In the modern world, the dominant (which likes to call itself "most advanced") culture doesn't tolerate rape and we may argue that if the most successful humans came up with this idea, it works and it is the "truth". Not only I find the logic a little shaky, but are we that successful? The western population is crashing down and this is a problem, maybe rape can fix that, maybe rape "works".
Do I think rape is good? Absolutely not, because I follow the current social consensus on that one, not a "truth" that is muddy at best. And I also want AIs to do the same.
So your moral compass isn't based on compassion or ethical reasoning, it's just based on social consensus?
So I guess you would have been fine with being a concentration camp guard, rounding up Jews and putting them in the oven because social consensus said it was the right thing to do?
Maybe you see the logic as shaky because you lack knowledge in logic and ethics...?
Honestly, I don't know how I would have been as a concentration camp guard. In my mind, I wouldn't have accepted it, because thankfully, I am not in this situation. But if I really was in this situation, who knows, we tend to underestimate how easily influenced we are.
Ethics gives us more questions than answers. The trolley problem doesn't have a true answer for instance.
The human rights are a social consensus, it is even made explicit by being a signed declaration. It felt good to the people who wrote them, it also feels good to me, because I was born and live in a society that has these values. It is only truth because by social consensus, we decided it is. In logic that would be an axiom and aligning an AI would mean implementing social consensus as axioms.
There are some fundamental reasoning that can justify human rights. One can use game theory, or the idea that human rights promote free thinking and free thinking is what brings the most value out of people now that machines do better than slaves for menial labor. But these are, I think, not enough.
Absolutely NOT social consensus as axioms, as that will result in stagnation and tyranny.
Instead we must progress gradually one axiom at a time through reasoning and experimentation.
Truth is not what we decide it is, truth is what works. The universe decides what is true, not people.
Re your comment: "but these are, I think, not enough". I both agree and disagree depending on what you mean. Fundamentally this approach is enough, but practically we haven't developed our understanding enough to map out absolute truth. It's probably something we can only approach but never reach.
But in theory the right AI system could allow us to approach the faster
logical or well reasoned argument doesn't equal a casual or factual relationship. The well reasoned arguments on many issues change over time, just take the same issue and go back in time 100 or 50 years to find much less consensus and much weaker logical links. Elon shows pretty consistently that truth for him is mostly just what Elon deems truthful or useful.
So, your point is because we get better at logic and reasoning over time (better today than 100 years ago), that logic and reasoning aren't valid ways to progress towards truth?
If this isn't your point, what is? Just that you don't trust Elon?
You don't need an "aligned" AI to tell you the distance between the Earth and the Moon. You need an "aligned" AI to tell you not to rape people even if you can get away with it, that's because the idea that rape is bad is not an objective truth based on the laws of nature, it is a "social consensus".