But at what point do 'consequences' start to negate the 'free' in speech?

I'd argue you have reached the limits of free speach the moment there are consequences for just the speech.