I'm curious what you folks think of things like Racket and it's ability to create DSLs for various problem spaces. This seems like it would let you stay in one language yet have the flexibility to still create the right tool for specific jobs.
The same goes with the abilities of its macros too. Does that give you the ability to define what you want and have the language create the code for you?
Caveat, I'm new to programming and just now learning Racket so it has me all excited but I barely know what I'm talking about. This thread has been very interesting to read so I'm curious what others think of languages which have the flexibility to adapt. Maybe it creates a maintenance or understanding other people's code nightmare or something. I'm just curious why Racket's approach or something like it hasn't taken off.
I don't have much experience with Racket, but I've read a lot of the papers. I think they're exploring exactly the right problems.
I'm not convinced Lisp is the best substrate for it. It makes more choices than you think (number representations, etc.) -- it's not completely neutral.
S-expressions turn out not to be the lowest common denominator -- strings are! "Everything is a file" in Unix means that everything is a big lump of bytes -- e.g. as opposed to the early Mac file systems which had metadata too.
The web is very much an extension of the Unix philosophy -- composing languages, ad hoc string manipulation, etc. with HTML/JS/CSS and dozens of other DSLs. Of course, this architecture has a lot of downsides, and that's what I'm working on in Oil.
-----
Other anecdotes:
There's a research shell called "Shill" built on top of Racket. I heard that they moved off of it for some reasons related to Racket's runtime.
Also, I heard that Racket is thinking of replacing their runtime with Chez Scheme (which is one of the more impressive Scheme runtimes as I understand, being in industrial use for 20+ years.)
So I think that Racket is good within its own world, its own runtime, but weaker when it has to interoperate with the rest of the world. Unix and the web are the rest of the world... and they don't speak s-expressions, so s-expressions really offer no advantage in that case.
-----
If you think about it, Unix already has the same architecture as Racket. It's a bunch of DSLs (shell, make, C, C preprocessor, linker scripts, etc.) on the same runtime (the kernel, which talks to the hardware). It's just that you are forced to parse everywhere and serialize. That is definitely annoying, and problematic. But empirically, that design seems to be more evolvable / viral / robust.
Anything you can do with macros, you can also do with a traditional lexer/parser/compiler. Macros make things easier, but conceptually the architecture is the same.
Parsing is difficult for sure, but I do think that most programmers have an unnecessary aversion to it. I hope to tackle this problem a bit with Oil, i.e. make it easier to write correct, fast, and secure parsers.
Anyway that's my take, hope it helped!
EDIT: I should also say to not take this as discouragement from learning Racket. My first college class was SICP and it had a huge and permanent effect on my thinking. (I learned from reading Racket papers that my TA went on to work on Racket itself.)
I just didn't use Lisp for any industrial job thereafter. But that doesn't mean the experience wasn't valuable. I would definitely learn it for the ideas.
The same goes with the abilities of its macros too. Does that give you the ability to define what you want and have the language create the code for you?
Caveat, I'm new to programming and just now learning Racket so it has me all excited but I barely know what I'm talking about. This thread has been very interesting to read so I'm curious what others think of languages which have the flexibility to adapt. Maybe it creates a maintenance or understanding other people's code nightmare or something. I'm just curious why Racket's approach or something like it hasn't taken off.