I agree, that's the main issue with this approach. Long-term, it should only be used for truly sensitive actions. More mundane things like replying to emails will need a better solution.