I wonder if the benefit is primarily for transactional vs analytical queries

it'll be purely for analytical queries

How does this compare with delta lake and iceberg?

Vortex is a file format, where as delta lake and iceberg are table formats. it should be compared to Parquet rather than delta lake and iceberg. This guest lecture by a maintainer of Vortex provides a good overview of the file format, motivations for its creation and its key features.

https://www.youtube.com/watch?v=zyn_T5uragA

The website could use a comparison / motivation in comparison to Parquet (beyond just stating it's 100x better).

Agreed, really need a tl;dr here, because Parquet is boring technology. Going to require quite the sales pitch to move. At minimum, I assume it will be years before I could expect native integration in pandas/polars/etc which would make it low effort enough to consider.

Parquet is ..fine, I guess. It is good enough. Why invoke churn? Sell me on the vision.

DuckDB just added support for vortex in their last release using the Vortex Python package so hopefully other tools wont be too far behind

> Going to require quite the sales pitch to move.

Mutability would be one such pitch I would like to see ...

I think it would still make sense to compare with those table formats, or is the idea that you would only use this if you could not use a table format?

That’s like comparing words with characters.

Vortex is, roughly, how you save data to files and Iceberg is the database-like manager of those files. You’ll soon be able to run Iceberg using Vortex because they are complementary, not competing, technologies.

As others said, Vortex is complementary to the table Formats you mentioned.

There are other formats though that it can be compared to.

The Lance columnar format is one: https://github.com/lancedb/lancedb

And Nimble from Meta is another: https://github.com/facebookincubator/nimble

Parquet is so core to data infra and widespread, that removing it from its throne is a really really hard task.

The people behind these projects that are willing to try and do this, have my total respect.

The title makes it sound like we have reached the singularity. The real insight here is that amateurs may have a difficult time competing with AI

I agree but if this year the AI can keep up with amateurs, next year who knows?

If you got married this month you will have 12 wives in a year, who knows?

MAXSTACK: Web framework for rapidly building SaaS apps with AI - trying to enable the next wave of 'fast-fashion saas'. Think of it like better-auth is doing for auth, I want to do for the rest of SaaS

- comes with common SaaS features pre-built (crud, blog, auth, etc.) - import templates from the framework until you want to customize them - create forms with just a zod schema - good docs, typescript interfaces, a CLI for common tasks, and MCP for your AI agent

If you're building something now or want to - I'd love to help. Could use the experience to make things easier through my framework.

I wonder how much we need this vs implementing it as part of Delta Lake or Iceberg

yeah, I think you could get much of the benefits if Iceberg (or ducklake) added support for `.vortex` (e.g. ducklake only supports `.parquet` right now)

I'm glad there's vibe coding available to non-tech founders so they can build the equivalent of high-fidelity prototypes. May not be MVP worthy yet, but at least it gets them started.

In reality what it will more likely result in is for them to get a false feeling of competence and they'll start to think that they don't need those tech folks anymore, even more than they already do.

They'll put even less value in the people doing the implementation and double down on them being the person with the vision and that's what makes them so great.

There was an intermediate step where they introduced AI command generation (super useful). Agentic coding follows naturally from that.

SaaS vendors do these business value assessments, which are useful to the executive buyer and the vendor. The issue I find with them is customers don't want to do them in collaboration with the vendor since the vendor will use that to justify higher prices. 'Outcome' and 'business value' seem somewhat synonymous - so maybe need to focus on the business value more closely (and all the modelling that goes into that). I don't know if all of the billing needs to go through this, but perhaps discounts or bonuses could - at the very least helping with retention.

Glad to see focus being put on keeping humans in the drivers seat, democratizing coding with the help of AI. The syntax is probably still too verbose to be easily accessible, but I like the overall approach.