Yeah, too often. But that's not the toggle's fault. The majority of the time the design (colors) and the associated label are the root problem. A toggle doesn't have to be ambiguous. Unfortunately, designers / UX'ers seem to think so.