> But real users don't talk like that - they ask loaded questions like "How is X not in jail?"
If the model can answer that seriously then it is doing a pretty useful service. Someone has to explain to people how the game theory of politics works.
> My study asked: where does the model actually land when it takes positions? A model can score 95% on even-handedness (engages both sides well) while still taking center-left positions when pushed to choose.
You probably can't do much better than that, but it is a good time for the standard reminder that left-right divide don't really mean anything, most of the divide is officially over things that are either stupid or have a very well known answer and people just form sides based on their personal circumstances than over questions of fact.
Particularly the economic questions, they generally have factual answers that the model should be giving. Insofar as the models align with a political side unprompted it is probably more a bug than anything else. There is actually an established truth [0] in economics that doesn't appear to align with anything that would be recognised as right or left wing because it is too nuanced. Left and right wing economic positions are mainly caricatures for the consumption of people who don't understand economics and in the main aren't actually capable of assessing an economic argument.
[0] Politicians debate over minimum wages but whatever anyone thinks of the topic, it is hard to deny the topic has been studied to death and there isn't really any more evidence to gather.
If the model can answer that seriously then it is doing a pretty useful service. Someone has to explain to people how the game theory of politics works.
> My study asked: where does the model actually land when it takes positions? A model can score 95% on even-handedness (engages both sides well) while still taking center-left positions when pushed to choose.
You probably can't do much better than that, but it is a good time for the standard reminder that left-right divide don't really mean anything, most of the divide is officially over things that are either stupid or have a very well known answer and people just form sides based on their personal circumstances than over questions of fact.
Particularly the economic questions, they generally have factual answers that the model should be giving. Insofar as the models align with a political side unprompted it is probably more a bug than anything else. There is actually an established truth [0] in economics that doesn't appear to align with anything that would be recognised as right or left wing because it is too nuanced. Left and right wing economic positions are mainly caricatures for the consumption of people who don't understand economics and in the main aren't actually capable of assessing an economic argument.
[0] Politicians debate over minimum wages but whatever anyone thinks of the topic, it is hard to deny the topic has been studied to death and there isn't really any more evidence to gather.