CoCs didn't jump out of the void for no reason. They are a response to years of harassment and unprofessional behavior at conferences. Conferences have always had a list of rules to follow and breaking the rules could get you ejected from the conference. In theory all a CoC should do is codify some bad behaviors that people believe to be acceptable but are actually not, so in that regard they shouldn't be controversial.
You hit the nail on the head though: if the people enforcing the conference rules don't act in good faith or don't follow the rules then the rules don't matter.
In theory a conference could just have a rule that says "act professionally" and if properly enforced that would be sufficient. On the flip side such a conference could have a petty tyrant rules lawyer for an enforcer and decide anyone they don't like was not acting "professionally".
There is no outside body that could (or should) be adjudicators. Conferences live and die by their reputation. All we can do is judge them by their actions.
I think I agree with your points on the surface, but disagree with (what I presume to be) your implications. It sounds like you're arguing that the current system is somehow a step forward. I very much question that.
To your point, CoCs are not really the issue. The issue is that CoCs and their subsequent enforcement are often advanced by ideological zealots engaging in witch hunts. I don't see how that is in any way an improvement over the status quo. I see no evidence that suggests these zealots are able to effectively deal with real harassment when it happens, either.
You hit the nail on the head though: if the people enforcing the conference rules don't act in good faith or don't follow the rules then the rules don't matter.
In theory a conference could just have a rule that says "act professionally" and if properly enforced that would be sufficient. On the flip side such a conference could have a petty tyrant rules lawyer for an enforcer and decide anyone they don't like was not acting "professionally".
There is no outside body that could (or should) be adjudicators. Conferences live and die by their reputation. All we can do is judge them by their actions.