This is something we talk about all the time here at Mozilla Research. Languages do not become popular primarily because of technical innovation; they become popular because they were in the right place at the right time. (You might draw an analogy to startups here.) Often the designers of the popular languages didn't have much of a background in programming languages research, so they didn't break new ground. (That's not to say that the current languages aren't good, though. On the contrary, Python, Ruby, Perl, etc. are all awesome languages! They just weren't particularly innovative from a PL design standpoint.)
Given that, what is the place for PL research in industry? There are two main ways that we can see for PL research to make a practical difference:
(1) Apply PL research to new features of existing popular languages, and push for those features to be standardized if they're successful. This is what we're doing with JavaScript with, for example, the PJs project (parallel extensions for JavaScript). It's always trickier to fit new research into paradigms of existing languages, but if it can be done it's very rewarding. This has been done with e.g. Java in the past; generics and the flow-sensitive initialization checking were an example of this.
(2) Create new languages designed for a specific, concrete high-value project. This is what we're doing with Rust -- Mozilla is investing in Rust specifically to be the language that we write a new parallel layout engine in. This constraint helps us focus on keeping the language usable, practical, and feature-rich. Features like the uniqueness typing and (in the future) bounded task lifetimes that allow us to avoid concurrent GC are driven by the pain points that we struggle with in Gecko.
I can’t remember the exact quotation, but it goes along the lines of, “Every successful language began as the scripting language for something popular.” (Anyone able to provide the correct quote?)
If that’s the case, “The right place at the right time” is when something new is taking off, like C for Unix, Ruby for Rails, Objective C for iOS, or recently JavaScript for Node.
Given that, what is the place for PL research in industry? There are two main ways that we can see for PL research to make a practical difference:
(1) Apply PL research to new features of existing popular languages, and push for those features to be standardized if they're successful. This is what we're doing with JavaScript with, for example, the PJs project (parallel extensions for JavaScript). It's always trickier to fit new research into paradigms of existing languages, but if it can be done it's very rewarding. This has been done with e.g. Java in the past; generics and the flow-sensitive initialization checking were an example of this.
(2) Create new languages designed for a specific, concrete high-value project. This is what we're doing with Rust -- Mozilla is investing in Rust specifically to be the language that we write a new parallel layout engine in. This constraint helps us focus on keeping the language usable, practical, and feature-rich. Features like the uniqueness typing and (in the future) bounded task lifetimes that allow us to avoid concurrent GC are driven by the pain points that we struggle with in Gecko.