I've structured my life such that the type of work I do serves my broader life goal.

At the point in which that's not possible, then how I spend my time is out of my control and the question becomes moot because I'd effectively have no control over my life.