Rather than fascism per-se, actions like holding people without charge, perusing their personal belongings and papers without due process, and threatening families or associates, are the marks of totalitarianism. It exists on both the right and left ends of the political spectrum.
Overusing these labels dilutes their 'brand', and then you don't have appropriate adjectives for real cases of fascism or genocide.