> Some individuals, particularly those running Linux, follow a standard called XDG Base Directory Specification. While Murex does not adhere to this standard, instead conforming to the de facto standard defined by the past precedents of previous shells, in order to offer flexibility for those who do prefer the XDG specification Murex does support overriding its own default paths via special environmental variables.
no no no you're missing the point: on POSIX systems, go with XDG by default. cmon guys it's 2023 not 2003
Murex doesn't just target POSIX systems and XDG added a lot of edge case problems I wasn't prepared to deal with at the time. Particularly so back when the shell was first created. So I went with the de facto shell standard instead.
You can override that behaviour if you want to use XDG but making it the default could break backwards compatibility, result in confusing documentation for non-POSIX users and introduce a lot of additional development and testing just to fix something that already works fine.
Like with any open source project, if someone else is willing to commit some time into solving these concerns then I'll gladly merge it. But I need to be pragmatic with how I prioritise my development time.
Every time I see one of these new shells, I'd like to know - how does it compare against the existing higher-end shells like Fish? What extra features does it have, which are missing, where are the differences?
If it offers substantial benefits over Fish, then I'd gladly take a look.
As it stands, the description is quite brief and doesn't talk as much about other shells and feature comparison.
I very intentionally kept the README like this because I didn't want to be seen as starting shell flamewars. Plus writing promotional material isn't my strength.
Improving this side of things is something that's constantly on my mind though. Plus if anyone else also wanted to contribute then I'd welcome that with open arms.
This is outstanding, personally I would immediately switch to it if it was an addon for zsh instead of a standalone new shell.
While I'm spending quite some time in my shell and the extend that its a core part of my system is quite considerable and makes switching to another shell altogether a huge time investment.
Nonetheless, I'll always keep an eye on promising alternatives and eventually switch over once the functionality missing gets apparent enough on a constant basis as it happened for me when switching from fish to zsh.
> once the functionality missing gets apparent enough on a constant basis as it happened for me when switching from fish to zsh.
As someone that went the other way ~5 years ago, I’m curious what made you switch :). Are you btw on a hand-rolled config or one of the distributions?
(For me, it was that I was maintaining a ~400 line zsh config, and then learned that fish did most of the things I had configured “out of the box”. I considered switching back for increased POSIX compatibility since I often write bash scripts at work, but since Fish 3 added POSIX-compatible syntax for most pain points, I haven’t kept up with zsh.)
This naming conflict is actually one of my regrets within Murex. I was a couple of years into development on this shell before I obtained a mac test system. I think most of the Murex users in the early days were also Linux users too. So I wasn't aware of the conflict until there was already too much code written in Murex to be worth the risk renaming the builtin.
However can mitigate this by running `exec open ...` -- you can set that as an alias as well, eg
Whereas on Linux it is a shell alias. Doesn't appear to be universal either (doesn't exist on Arch, for example). It was a recent change too, added to Debian 11 (released 2021), which is years after it was added to Murex.
As an aside, I wouldn't be at all surprised if the `open` alias was added to Debian to copy (and I mean this affectionately) macOS. Otherwise `xdg-open` might have just been called `open` from the beginning.
> As an aside, I wouldn't be at all surprised if the `open` alias was added to Debian to copy (and I mean this affectionately) macOS. Otherwise `xdg-open` might have just been called `open` from the beginning.
xdg-open was called xdg-open because the name open was already taken by another Linux command - to run a program on a new virtual console. However, since virtual consoles are used far less than they once were, eventually people decided to reserve the obvious name for the common function - so open got renamed to openvt, and open became an alias for xdg-open. It also helped prevent the confusion for people coming from macOS to Linux, trying to use open and getting a completely different command instead, and then asking “why do I have to put xdg- first???”
There is the same command function wise on Debian Linux. Alias or not makes no difference.
Debian Linux is the base of almost all Desktop Linux boxes out there. (There are of course niche distris, but the overwhelming majority is Debian based; I count Ubuntu as Debian based.)
There is nothing "specific" to my shell environment. Having the `open` alias available the default. I would need to change things manually to diverge form this default. That would be "specific to my shell environment", not the other way around.
Historically `open` comes form NextStep, so actually Apple copied it.
On Linux it's usually a symlink as there are other implementations then the one in `xdg-utils`, and there have been also other programs with that name in the past.
First of all you describe it as an alias, which is very different to a symlink. A symlink is a file so will be available system wide. Whereas an alias is only available to shells that have had that alias added to (and it's pretty common for embedded shells, like in IDEs, to not pick up default aliases). So my points were about aliases not symlinks.
For what it's worth "open" doesn't exist on my Ubuntu servers either. And it was added to Debian only 2 years ago. Murex has been around for nearly a decade. So I can hardly be blamed for adding a builtin with the same name as something that hadn't yet been added to Linux.
> Historically `open` comes form NextStep, so actually Apple copied it.
Apple didn't copy NextStep, they bought it and ported/rebranded a lot it's tech. Also I wasn't claiming Apple invented the concept of "open". I was saying I suspect Debian got the idea from macOS.
---
Anyway, this is all moot because the `open` builtin (Murex) already checks if `open` is a TTY (ie you're not piping the output of `open`). You can define how commands get opened when it's a TTY as well https://www.murex.rocks/docs/commands/openagent.html -- this is how Murex can inline images into the terminal -- so i can just add a "if no open agent is defined fallback to exec open" type condition. Then everyone is happy.
As to why `open` builtin reinvents `xdg-open` -- well that's because Murex needed something cross platform and there wasn't a reliable way to do that at the time. So I had to roll my own. Now the landscape has changed somewhat, I daisy chain the builtin to call the system `open` command on supporting platforms.
Maybe. But I just don't know why this discussion started in the first place. There is a `open` command on likely almost every desktop Linux box. So it's not Mac specific (which was my sole point in my first comment in this thread).
> Right, you said "alias" multiple times earlier. Symlinks and aliases are not the same thing.
You're technically correct. My fault!
I've called it alias as it's a symlink coming form the Debian alternatives facility. The alternatives mechanisms manages system wide command "aliases"… I should have been more exact in this point.
> I was saying I suspect Debian got the idea from macOS.
I have no prove but I doubt that. There were other tools on Linux called `open` before MacOS X came along as far as I know. The whole thing seems to be rooted in BSD (where NextStep got it's Unix parts form).
> For what it's worth "open" doesn't exist on my Ubuntu servers either.
Makes perfect sense. It's a desktop tool.
> And it was added to Debian only 2 years ago.
I don't think so. I had `xdg-utils` and it's `open` command installed for many years. It's at least 17 years old:
I'm running Debian Testing so I had this command in fact since many years. Didn't know that it's actually quite "new".
Now I can understand how it could happen that Murex created a command name conflict. External tools don't test against Debian Testing most of the time…
Note how the line ends with a backslash. That is escaping the newline so the command isn’t executing immediately. The next line begins with a pipe which is taking the input. It’s a way of separating multiple piped commands over several lines for readability. In other words, these are equivalent:
You can just end each partial line of such long commands with a pipe sign and keep writing the rest of the overall command on subsequent lines, except for the last partial line, where you just press Enter to complete and run it.
Reminds me somewhat of nushell[0].
It's mindblowing what's possible with it. It makes the oneliners you normally write more readable, but I'm still using a "normal" shell (zsh) as I can't use these everywhere I want, compared to most of the oneliners that only depend on coreutils and can be run on any gnu/linux system.
If I'd need to install another tool to take my scripts with me, I'll probably would just use another scripting language like Python.
Nushell is pretty awesome but the one problem with Nushell is it requires commands to "speak" nushell for it's magic to work. Take the example on the landing page with `ls`. `ls` is actually a nushell builtin rather than GNU / BSD / whatever coreutils: https://www.nushell.sh/commands/docs/ls.html
Whereas Murex colours pipelines with type annotations so that it can perform it's magic with your existing system's coreutils.
That's not to say that Nushell's approach is wrong. It's a popular approach used by a few alt shells, including PowerShell. But it does come with trade offs:
1. your coreutil muscle memory needs to remember whether you're in Nushell or Zsh (eg `ls` flags will differ depending on your shell)
2. the developers of Nushell need to re-implement coreutils to expose the genius of their shell. And as we've seen on here before, developing and maintaining coreutils is far from a trivial task
3. any other CLI tools that people might use that might not be common might also not support Nushells magic
These problems doesn't exist for Murex because it is basically an abstraction on top of POSIX.
To be clear though, I'm not making a criticism about Nushell. Murex has it's own problems and trade offs as well. So please take this comment only as a description of where our approaches have differed (I think it is great that there are numerous options these days).
It should be noted that `ls` is kinda of a (interactive convenience) special case by itself, if you really need to parse `ls` output, you're in a world of hurt on any conventional raw_byte_stream+pipes based shell [1]. Most (non-interactive) tasks done with ls should instead be done with other, safer shell builtins or more appropriate tools.
I think that's a good comment on Nushell. I agree that it's doing lots of things right and is at the very least an excellent proof-of-principle regarding how our shells _should_ behave, but at a very high cost of lost interopability.
> the developers of Nushell need to re-implement coreutils to expose the genius of their shell.
Yes, or if Nushell is going to try not to be interoperable with existing POSIX commands, then the project needs to focus much more on perfecting the design of its exposed library of builtins.
As long as the command supports some structured output, such as json, it is very easy to get back the magic.
I have written a bunch of scripts that basically manages my whole home server using podman by leveraging `podman --output=json`. Most new tools support json output that make it easy, and by adding an alias that adds the `--output=json | from json` (or the equivalent for each command) it works pretty much the way you would expect.
For others that don't, you just need to add a little parsing to kick it off. Here are some examples on their github[0]. Once you have what works, just add it as an internal command, and it's a "fixed" problem.
I personally prefer nushell's method, as it allowed me to add tools that do some more advanced stuff pretty quickly.
You can do that in Murex too. In fact it's even easier than the examples given because Murex expects STDOUT to be structural data by default. You just need to tell it what the format is if the pipeline isn't type cast by default.
I don't have Windows to test Windows `netcat` on but the Murex code would look something like this:
netstat -ao -p tcp -b | [Proto..]r | tabulate --separator " +" | [Proto "Local Address" "Foreign Address" State PID]
# [Proto..]r -> select every line after the regexp expression "Proto"
# tabulate --separator " +" -> by default columns are split on whitespace. But here we are saying use two or more spaces
# [Proto "Local Address" "Foreign Address" State PID] -> selects columns (we already have column titles from `netstat` so why reinvent the wheel?)
Likewise with your `podman` examples, in murex this would look like:
function podman {
cast json
exec podman --output=json @PARAMS
}
You can even configure Murex REPL to only autocomplete commands that support JSON input from `podman`:
method define podman %{ Stdout: json }
...so now when you type `podman | <tab>` you only see commands that are compatible with JSON.
I do have a lot of respect for Nushell but I've been using Murex as my primary shell for longer than Nushell has been around so a lot of these edge cases have been solved in Murex too. I just don't do a particularly great job at advertising it :)
Fair enough. Though the difference between your netstat examples and the podman examples and the nushell examples are minimal at best. Even if I understand the difference of the implementation (between "coloring" the output, and parsing it).
Yeah. This is a little half-baked, so bear with me....
It seems to me like tools like bash, lmorg, nushell, and PowerShell exist in some kind of continuum (which I think is, from left to right as I've indicated them above, the amount of "smarts"/type information objects returned from commands have). Perhaps not coincidentally, the further right you go, the more what you have starts looking like a programming environment than a shell to execute other programs.
I've used zsh for many years, and for me the biggest wins were the various globbing enhancements that all but removed the need for `find`. I know bash has caught up on a lot of these but now it's just momentum.
I also still script in bash, as zsh sometimes has some minor deviations that are not directly that obvious.
One of the reasons I use zsh is because of all the plugins that exist for it. With them I have the feeling that they boost my productivity. But the real reason is, zsh feels more like it does what I want/expect compared to bash (, that is probably subjective?).
You might like https://github.com/sharkdp/fd. I use it as it is fast, although I've used `find` for so long I have to think about the differences before I do.
On topic of alternative shells: recently I learned about rash[1] and I’ve been absolutely delighted. I really wish it’d gain some traction as the project seems to be moving very slowly (if at all)
Apart of the .NET/Microsoft/ugly command-lets situation, how do you pitch yourself against PowerShell? The typed pipelines and advanced programming are also there.
Not that I am a personal Powershell fan (in-memory commands and ugly commands is the ugly part for me), but "next-gen" shells with typed streams are not exactly the novelty.
Murex is an abstraction on top of POSIX rather than a replacement for POSIX and coreutils et al. Which means Murex works with all your existing CLI tools without you having to worry altering your muscle memory.
If people are already using and like Powershell then I'd recommend they stick with Powershell. But if anyone is using Bash, Zsh or Fish (for example) but longed for something a little more intuitive (fewer footguns, smarter handling of structural data, etc) then Murex is a good option.
It's also worth noting that I've modelled the interactive shell on IDEs rather than readline. So the UX in Murex is, in my biased opinion, one of the best out there. However I'm always open to feedback on anything that can be improved in that department. Or in fact, anything regarding the shell.
Did not realize that POSIX after it was obsolete standardized commands ;) anyway.
But I get you. PowerShell is intended to harmonize server management and yours is a general purpose shell. Sounds like a very reasonable differentiation.
Do you write articles about how you make coreutils output structured? Is that a command specific adapter or how does that work?
> Did not realize that POSIX after it was obsolete standardized commands ;) anyway.
POSIX does standardise some commands[0], albeit not many and good few of them are implemented as builtins even in Bash. But it also defines STDOUT (etc) files. The way Powershell passes data is as .NET objects. The way Murex passes data is as POSIX byte streams but with type annotations passed out of band. This creates some extra overhead in serialising data but the advantage is that it works with all of your existing tooling without modification.
> Do you write articles about how you make coreutils output structured? Is that a command specific adapter or how does that work?
I haven't but that's a good idea.
The way it works is just that you add type annotations. The default type is assumed text data, potentially in columns, like you'd see from `ps aux` or `ls -l`. There are also some builtins (eg [1][2]) that are data type aware.
So if you just use the most basic features of Murex, you get a few footguns removed (eg by default variables containing spaces don't get split into multiple parameters) and a slightly different syntax to Bash but all the same CLI tools work. However if you want to invest more into Murex then you get greater power from it eg the ability to inline SQL queries[3], write unit tests[4], better error handling[5][6], etc as well as being able to do `jq` like expressions against a multitude of object and table formats.
I really do need to get better at publishing it's features though!
One thing I enjoy about fish is that it's not just a shell, but also a modern scripting language alternative to sh/bash. Seems like I would lose this with Murex?
(There's no right or wrong choice for Murex here, just that has to choose one way or another.)
It's a completely new scripting language. It uses C-style braces rather than the more ALGOL-style `if / fi` keywords. However the core syntax of the language should still be familiar to those who use Bash (eg pipes, commands, `$` sigils).
So there is a little more to re-learn than you would with Fish but it shouldn't feel alien to read and write.
zsh has completion for kill, pkill, etc. out of the box. The default for kill is to only shown processes from the current pty, but you can change that with something like:
zstyle ':completion:*:*:kill:*:processes' command 'ps xo pid,user:10,cmd'
I also filter some processes that I almost never want to kill to clean it a bit up:
As with most things in zsh, you can make it as fancy as you want, but this is enough for me.
zsh is really powerful, but the downside is that a lot of these things have a bit of a "magic incantation factor" unless you're pretty familiar with zsh scripting. I guess this is why people use oh-my-zsh and things like that.
Can't have it all I'm afraid. For this reason I keep my zshrc contained to one file so I can easily move it around, but it does require zsh to be available in the first place of course.
All things considered, I'd rather optimize for the 99% of the time I'm on my local machine than the 1% I'm on a remote machine.
Not the person you responded to, but I probably kill an octave or matlab terminal daily. Oops, it turns out this matrix is too big to solve directly. Oops, norm2 takes forever and norm1 would have been good enough.
I suspect they're talking about fzf in general, using `kill` as an example, rather than saying they `kill` stuff often. But I could have misinterpreted their comment.
My own, personal, use-case is to completely nuke `bash-complet*` off my machine.
To me, bash-completion is horrible. I am looking for a tarball to unwind, and I only remember that it it started with "foo-" and it sure did not end in ".tar" (trying to save keystrokes, y'know). Trying C-i does not pull up anything. Argghh. I am on a machine with bash-completion!
---
I often hit C-a then "a" to get an invalid command (still saved to my hist), or C-a C-k if I think there is nothing else going into the kill-ring. bash-completion absolutely messes with me.
If it works for You, that is great. I really dislike it.
This looks great. Say I’m writing a script or Python program, is there a way for it to know that the input has type “json” (and still work if the user didn’t install Murex)?
no no no you're missing the point: on POSIX systems, go with XDG by default. cmon guys it's 2023 not 2003