dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
dagster-io / dagster
apache / burr
Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.
astronomer / dag-factory
Construct Apache Airflow DAGs Declaratively via YAML configuration files
astronomer / astronomer-cosmos
Run your dbt Core or dbt Fusion projects as Apache Airflow DAGs and Task Groups with a few lines of code
OpSafari / laborix
Autonomous experiment-pipeline orchestrator: versioned research-plan DSL with dependency DAGs, deterministic checkpoint/resumable executor, tamper-evident provenance ledger, UCB1/Thompson schedulers with measured regret against synthetic oracles, ablation bookkeeping, offline report synthesis. NumPy core, CPU-only torch extra.
Unknown-333 / awesome-data-engineering-skills
36 data-engineering skills for AI coding agents — dbt, Airflow, Dagster, Prefect, Spark, Snowflake, BigQuery, Databricks, Kafka, Iceberg. Portable SKILL.md Agent Skills (the format Anthropic launched as Claude Skills), with idempotency, backfill, and data-quality know-how. Runs in Claude Code, Cursor, Codex, Copilot, Gemini CLI and more.