Yes, the ability to prohibit an employee from choosing who to work for, while costing the company nothing at all does indeed lead to worse outcomes for the worker.
Any non-compete clause must come with a requirement to pay that employee to not compete. That will quickly get rid of non-competes that aren't actually their for the purpose of protecting trade secrets.
On Wall Street we often have Garden Leave, where an employee continues to receive salary and benefits for a period of time after resigning to go to a competitor. Seems pretty fair to me.
Any non-compete clause must come with a requirement to pay that employee to not compete. That will quickly get rid of non-competes that aren't actually their for the purpose of protecting trade secrets.