The pyproject.toml, package.json, and Cargo.toml are declarative project configuration files. While the Rust community refers to Cargo.toml as a manifest, it is not a comprehensive and detailed list of a build. That is the lock file.

While go.mod does not allow for explicit version ranges, the versions given are the minimum versions. In other words, the versions given are the lower bound of the compatibility range.

Go also strictly follows semantic versioning. Thus the implicit exclusive upper bound is the next major version. This assumes that all minor and patch releases are backwards compatibile and not breaking.

Dependency resolution in Go uses minimum version selection. That means the minimum requirements of all dependencies are evaluated and highest minimums are selected. In principle, this minimum version selection should be time invariant since the oldest versions of the compatible dependencies are used

While the minimum versions specified in go.mod are not necessarily the version of the dependencies used, they can be resolved to the versions used irrespective of time or later versions of dependencies being released.

Other languages do not use minimum version selection. Their package resolution often tries to retrieve the latest compatible dependency. Thus a lock file is needed.

Python packages in particular do not follow semantic versioning. Thus ranges are critical in a pyproject.toml.

In summary, the "manifests" files that the original author describes are configuration files. In some languages, or more accurately their package management schemes, they can also be lock files, true manifests, due to version semantics. If those semantics are absent, then lock files are necessary for compatibility.

> While the minimum versions specified in go.mod are not necessarily the version of the dependencies used

This has not been true since Go 1.17 with the default -mod=readonly, which is why go.mod is a reliable lockfile.

That’s very interesting. Most systems I know would pick the highest versions allowed by the ranges. In maven and gradle, for example, at least by default they choose the highest versions allowed. Even if no version range is used, it picks the highest choice even across major versions, which I always thought was completely broken. What does go do if you have two transitive dependency versions whose allowed major is different?

In some sense, Go does not allow you to change the major version. Packages with the same name but different major versions are treated as different packages.

"overengineered" is not the term I would use to describe Python packaging. I would say it is "under-engineered". As in, "Why engineer a configuration file when you can just do it in code?".

This tendency towards what initially seems like the "simple" solution pervades the Python ecosystem and often requires complex engineering to work around later.

Summary: They fixed Python packaging by not having to run Python to resolve dependencies. Also, they used Rust.

Moral of the story: Use less Python. Use declarative configuration and other langauges instead.

>> you can literally write python wrappers of Julia compiled libraries like you would c++ ones. > Yes, please. What do I google? Why can't julia compile down to a module easily?

Try JuliaC for compiling shared libraries if that is what you mean by a "module": https://github.com/JuliaLang/JuliaC.jl

That said Julia's original design focused on just-in-time compilation rather than ahead-of-time compilation, so the AOT process is still rough.

> I don't understand why there's so much friction between julia and python. You should be able to trivially throw a numpy array at julia and get a result back.

You can throw a numpy array at Julia and get a result back. See https://juliapy.github.io/PythonCall.jl/stable/juliacall/

> I wish there was a way to specify the offset base with something like a C #define or compiler directive.

Julia has OffsetArrays.jl implementing arbitrary-base indexing: https://juliaarrays.github.io/OffsetArrays.jl/stable/

The experience with this has been quite mixed, creating a new surface for bugs to appear. Used well, it can be very convenient for the reasons you state.

  julia> A = collect(1:5)
  5-element Vector{Int64}:
   1
   2
   3
   4
   5

  julia> B = OffsetArray(A, -1)
  5-element OffsetArray(::Vector{Int64}, 0:4) with eltype Int64 with indices 0:4:
   1
   2
   3
   4
   5

  julia> A[1]
  1

  julia> B[0]
  1

I tried this in Julia with TidierData.jl, and it looks quite similar to the R version.

  using TidierData, DataFrames
  using PalmerPenguins: load

  penguins = load()

  @chain penguins begin
    DataFrame
    @drop_missing(body_mass_g)
    @group_by(species, island)
    @summarize(
      body_weight_mean =
        mean(body_mass_g),
      body_weight_std =
        std(body_mass_g)
    )
    show(_, allrows=true)
  end

> Unfortunately, there's no alternative, and I don't see R as much easier, there are plenty of ugly things as well there.

Have you tried Polars? It really discourages the inefficient creation of intermediate boolean arrays such as in the code that you are showing.

> There's Julia -- it has serious drawbacks, like slow cold start if you launch a Julia script from the shell, which makes it unsuitable for CLI workflows.

Julia has gotten significantly better over time with regard to startup, especially with regard to plotting. There is definitely a preference for REPL or notebook based development to spread the costs of compilation over many executions. Compilation is increasingly modular with package based precompilation as well as ahead-of-time compilation modes. I do appreciate that typical compilation is an implicit step making the workflow much more similar to a scripting language than a traditionally compiled language.

I also do appreciate that traditional ahead-of-time static compilation to binary executable is also available now for deployment.

After a day of development in R or Python, I usually start regretting that I am not using Julia because I know yesterday's code could be executing much faster if I did. The question really becomes do I want to pay with time today or over the lifetime of the project.

Yes, I tried Polars both in Python and Rust. In Python it's more verbose, as the other commenter points out. In Rust it's very painful.

> Have you tried Polars? It really discourages the inefficient creation of intermediate boolean arrays such as in the code that you are showing.

The problem is not usually inefficiency, but syntactic noise. Polars does remove that in some cases, but in general gets even more verbose (apparently by design), which gets annoying fast when doing explorative data analysis.

It sounds like there are many Python users who have acclimated to the situation of needing three or more tools to work with Python and do not see the benefit or value of being able to do this all with one potentially faster tool.

While I understand that some have acclimated well to the prior situation and see no need to change their methods, is there really no objective self-awareness that perhaps having one fast tool over many tools may be objectively better?

What tooling do you use?

The creators got burned on Swift for TensorFlow, their first MLIR project. One of the problems with that first venture under Google was that the language was not Python.