I think it's important to keep in mind that stereotypes about women not being as intelligent as well as black people not being as intelligent as non-black plays a (large imo) part in this as well.

Add the two together and there's an even higher likelihood the next person you see will think you couldn't possibly have the same major as them, are incompetent, didn't earn your way to your position, or generally don't belong.