It's not too hard to make a meta-manager, is it? Wrap the ten or so people use in a script, have dependencies like
"clib": {
// whatever
},
"gem": {
// whatever
}
So you could do something like install GSL in the C package and then Ruby GSL in your gems. Saves the trouble if installing things to bind to, lets the things manage what they manage best. Install for the meta-manager would install the relevant package managers, then you've got it all.
I think you significantly underestimate how difficult a problem correctly installing and configuring software is. Even just specifying and resolving the dependencies and relationships between libraries and packages is quite hard.
One thing I didn't realize until recently is that it's not just "hard" engineering-wise, it's hard theoretically, i.e. NP-hard. With version constraints, package managers actually have to solve a problem equivalent to SAT solving.
All the "practical" package managers rely on relatively terrifying hacks and heuristics to get around this. I have been looking through Debian metadata, and I am very surprised at how unsound it is. The metadata is very tightly coupled to apt-get ("virtual" package handling, etc.) Not to mention that all the algorithms basically only work in one direction (going forward, not backward).
It appears that some academic/niche package managers are incorporating SAT solvers.
"Managing the complexity of large free and open source package-based software distributions"
Anyway the root problem is that C headers and other language-specific interfaces are not always a good way to interface software components... developing more stable binary protocols would greatly ameliorate the versioning problem.
I was also originally surprised to realize that this problem is NP-hard. It makes sense though: it involves both universal (for all requirements & dependencies) and existential (there exist packages & versions) quantifiers.
Julia's package manager [1] actually uses a custom belief propagation algorithm [2] to find an optimal set of package versions to satisfy given requirements. Originally it used linear programming, but belief propagation is much more efficient and solves the problem optimally in all realistic cases we've tested it on. Using a SAT solver would be another approach, but there is still the issue that you don't just want to find one satisfactory solution, but the optimal one with the freshest versions of the least number of packages. Since SAT isn't an optimization problem, it doesn't give you that.
I have been working on productionization of R code, i.e. allowing people to ship their prototypes instead of rewriting algorithms in another language, so I am very interested in this problem.
I have also looked at lot at R's metadata format (CRAN), which, like Debian/Ubuntu, is very successful, despite being unsound.
It looks like METADATA.jl is versioned, while R punts on that... the version constraints in the "head" PACKAGES file are mutated regularly and it becomes impossible to install older versions of packages with correct dependencies.
I noticed in the docs there is a local package dir like: "/Users/stefan/.julia/v0.3". If this is settable for individual Julia processes rather than being a system-wide global, then I will be very happy :)
Pretty much all language-specific package managers started out with this as a global, and then as they evolved to "production quality", there were some truly horrible hacks layered on top: Python's virtualenv, Ruby's bundler, R devtools, etc.
I was a little skeptical about Julia at first (mainly the "one language for everything" philosophy), but I have seen lots of very impressive stuff so far.
This is definitely worthy of more attention/exposition, since it will be probably the first "principled" package manager in wide/"practical" use.
> I noticed in the docs there is a local package dir like: "/Users/stefan/.julia/v0.3". If this is settable for individual Julia processes rather than being a system-wide global, then I will be very happy :)
It's controllable via the JULIA_PKGDIR environment variable, so yes, it can easily be per-process; you can also easily change it while a process is running, which allows you to do operations on multiple different package directories.
> I was a little skeptical about Julia at first (mainly the "one language for everything" philosophy), but I have seen lots of very impressive stuff so far.
There's a subtlety to this philosophy: it's intended to be a language in which you can do everything, not in which you must do everything. It's very easy to call C, Fortran and Python from Julia.
Regarding the computational difficulty of dependency resolution, I ran into a pathological case with Bundler a while back [1]. The money quote from the EDOS folk is:
> automatic package installation tools ... live dangerously on the edge of intractability, and must carefully apply heuristics that may be either safe, and hence still not guaranteed to avoid intractability, or unsafe, thus accepting the risk of not always finding a solution when it exists.
In other words, you either risk running into pathological cases (as I did with Bundler) that will come up with a solution only after many hours (or more), or you use methods that terminate quickly but will not find every solution.
"clib": { // whatever }, "gem": { // whatever }
So you could do something like install GSL in the C package and then Ruby GSL in your gems. Saves the trouble if installing things to bind to, lets the things manage what they manage best. Install for the meta-manager would install the relevant package managers, then you've got it all.