dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.

pathwaycom / pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Unstructured-IO / unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

OpenDCAI / DataFlow

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

elementary-data / elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

mage-ai / mage-ai

🧙 Build, run, and manage data pipelines for integrating and transforming data.

fluvio-community / fluvio

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

ucbepic / docetl

A system for agentic LLM-powered data processing and ETL

Netflix / maestro

Maestro: Netflix’s Workflow Orchestrator

meltano / meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

feldera / feldera

The Feldera Incremental Computation Engine

bruin-data / bruin

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

quixio / quix-streams

Python Streaming DataFrames for Kafka

combust / mleap

MLeap: Deploy ML Pipelines to Production

opendatadiscovery / odd-platform

First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

elementary-data / dbt-data-reliability

This dbt package captures metadata, artifacts, and test results so you can detect anomalies, monitor data quality, and build metadata tables. It powers Elementary OSS and feeds the wider context layer used by Elementary Cloud’s full Data & AI Control Plane.