You're strawmaning the socialist view. The stealman version is that people who are feeling economic pain are more likely to want to do something about it, and may be primed to develop class consciousness and become politically mobilized. Socialists generally consider material conditions to be more important than identitarian concerns, which in their view, are often used as a wedge to divide working class people who might otherwise be united by their common economic interests. They don't think poor people are somehow magically less likely to be bigots.