Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I’m curious as to if anyone can say how this compares to dagster since both libraries seems to rely on deploying to engines like Airflow?


Kedro puts emphasis on seamless transition to prod without jeopardizing work in experimentation stage:

- pipeline syntax is absolutely minimal (even supporting lambdas for simple transitions), inspired by the Clojure library core.graph https://github.com/plumatic/plumbing

- sequential and parallel runners are built-in (don't have to rely on Airflow)

- io provides wrappers for existing familiar data sources, but directly borrows arguments from Pandas, Spark APIs so no new API to learn

- flexibility in the sense you could rip out anything, for example, the whole Data Catalog replacing with another mechanism for data access like Haxl

- there's a project template which serves as a framework with built-in conventions from 50+ analytics engagements




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: