The work we now consider 'important' is the work that has, according to our best guess, the highest likelihood of solving important problems in future.
Our best guess may be wrong, as the future is hard to predict. But the future is not so unpredictable that we should stop thinking about where best to direct our efforts.
Maybe someone working on Farmville will cause an important advance, as an unpredictable side effect. We can't rule that out. Even a job that seems 'unimportant' can have beneficial and unpredictable windfalls, and that's wonderful.
But we should still try and direct our efforts as best we can, and work on problems that we think are important.
That seemingly trivial pursuits sometimes bear fruit, doesn't mean that all pursuits are equally worthwhile.
The power of vast, organic, ecological systems like markets and large open source communities, is that we can work on all problems simultaneously. This is immensely powerful and it also allows network effects of problem solving, such as those side effects mentioned in the article. If nobody was working on the things that required big parallel clusters, then the advancements in GPU technologies would matter not a bit to solving these problems.
I don't think we need to worry about how important the problems we are trying to solve are. I think the best approach is to work on solving the problems that inspire us. I think, as the Spartan prince once said (according to Plutarch), "People raise positions, not the other way around."
Difficulty can be a decent heuristic for novelty, though. There are obviously a great number of simple things that have never been tried (or even thought of), but certainly difficult things are less likely to be tested, all other things being equal. That said, small ideas and small changes can be surprisingly impactful.
I rather like the idea of small changes that result in cognitive shifts. An article I read a while ago said that Lisp programmers are often unimpressed with language features other people have because Lisp can easily emulate and subsume them, essentially dismissing the idea for its technical simplicity and not an actual lack of merit.
I'm not a big web guy, but a lot of web tools seem to work that way - easy to think up, easy to code up, but once they're actually available they change the game in subtle ways. There's a mindset where people (myself included) say "Well, that's easy enough to hack up using cron and ssh and..." and we fail to recognise that actually having it makes us use it, and actually using it improves our lives.
Our best guess may be wrong, as the future is hard to predict. But the future is not so unpredictable that we should stop thinking about where best to direct our efforts.
Maybe someone working on Farmville will cause an important advance, as an unpredictable side effect. We can't rule that out. Even a job that seems 'unimportant' can have beneficial and unpredictable windfalls, and that's wonderful.
But we should still try and direct our efforts as best we can, and work on problems that we think are important.
That seemingly trivial pursuits sometimes bear fruit, doesn't mean that all pursuits are equally worthwhile.