apache / kafka
Apache Kafka - A distributed event streaming platform
apache / kafka
DataTalksClub / data-engineering-zoomcamp
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
debezium / debezium
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
schumannx / full-stack-data-engineering
End-to-end streaming data platform on AWS: Kafka → Iceberg lake → Kimball model → Athena, orchestrated with Airflow 3. Terraform, Glue PySpark, dbt.
redpanda-data / redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
twmb / franz-go
franz-go is a feature complete, pure Go library for Kafka from 0.8.0 through 4.4+. Producing, consuming, transacting, administrating, etc.
kafbat / kafka-ui
Open-Source Web UI for managing Apache Kafka clusters
openmeterio / openmeter
Metering and Billing for AI, API and DevOps. Collect and aggregate millions of usage events in real-time and enable usage-based billing.
Syngnat / GoNavi
High-performance multi-data-source database client — ~30MB, AI & MCP ready, zero Electron bloat. | 高性能多数据源数据库客户端：约 30MB，AI 与 MCP 就绪，告别 Electron 膨胀。
influxdata / telegraf
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
AutoMQ / automq
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
risingwavelabs / risingwave
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
apache / camel
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
ag2ai / faststream
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
svix / svix-webhooks
The open source and enterprise-ready webhooks service 🦀
beianchuqing111 / drone-fire-platform
面向无人机机场的实时视频 AI 火情检测服务：Kafka 任务调度 → 直播流多模型检测 → 地理定位 → 向量去重 → MQTT 报警
memgraph / memgraph
High-performance open-source in-memory graph database for GraphRAG, AI memory, agentic AI, and real-time graph analytics. Cypher-compatible, built in C++.
redpanda-data / console
Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.
akvorado / akvorado
Flow collector, enricher and visualizer
Graylog2 / graylog2-server
Free and open log management
ArroyoSystems / arroyo
Distributed stream processing engine in Rust
pmacct / pmacct
pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
datacontract / datacontract-cli
Open-source data contract testing for Snowflake, Databricks, BigQuery, Kafka & 15 more. Built on the Open Data Contract Standard (ODCS). MIT.
netsampler / goflow2
High performance sFlow/IPFIX/NetFlow Collector
TharunKumarReddyPolu / Go-Handbook-for-Software-Engineers
A comprehensive Go handbook for software engineers covering fundamentals, idiomatic Go, concurrency, generics, testing, APIs, databases, performance, microservices, system design, and production engineering.
asyncapi / spec
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
tianshiyeben / wgcloud
Linux运维监控工具，支持系统硬件信息，内存，CPU，温度，磁盘空间及IO，硬盘smart，GPU，防火墙，网络流量速率等监控，服务接口监测，大屏展示，拓扑图，端口监控，进程监控，docker监控，日志监控，文件防篡改，数据库监控，指令批量下发执行，web ssh，Linux面板(探针)，告警，SNMP监测，K8S，Redis，Nginx，Kafka，资产管理，计划任务，密码管理，工作笔记
conduktor / kafka-stack-docker-compose
docker compose files to create a fully working kafka stack
cruise-control-for-kafka / cruise-control
Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
warpstreamlabs / bento
Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.
robustmq / robustmq
Communication infrastructure for the AI era — one binary, one broker, one storage layer, any protocol
arkflow-rs / arkflow
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
vdesabou / kafka-docker-playground
🐳✨ Fully automated Apache Kafka® and Confluent Docker based examples // 👷♂️ Easily build examples or reproduction models
redpanda-data / kminion
KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.
shining-stars-l / javaup
🚀 超级八股文 🚀 - 告别死记硬背的 Java 全栈技术知识库！100万+字的讲解，原理剖析 + 丰富代码示例 + 清晰流程图解，让你真正理解每一个技术点！
confluentinc / confluent-kafka-python
Confluent's Kafka Python Client
RayElg / kafgres
Kafka broker embedded in postgres
PicoMQ / picomq
Kafka and Durable streams on object storage
redpanda-data / connect
Fancy stream processing made operationally mundane
HariSekhon / DevOps-Bash-tools
1200+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..
dotnetcore / CAP
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
strimzi / strimzi-kafka-operator
Apache Kafka® running on Kubernetes
syslog-ng / syslog-ng
syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.
rsyslog / rsyslog
High-performance log ingestion and ETL engine
php-rdkafka / php-rdkafka
Production-ready, stable Kafka client for PHP
confluentinc / examples
Apache Kafka, Apache Flink and Confluent Platform examples and demos
aklivity / zilla
🦎 A lightweight, streaming-native, multi-protocol gateway for AI, APIs, and real-time data. Route LLM requests across providers, govern MCP tools, and connect Kafka, MQTT, and HTTP through one high-performance engine with shared security and observability.
quixio / quix-streams
Python Streaming DataFrames for Kafka
java-up-up / damai
🔥 🔥 官方推荐 🔥 🔥 高并发大麦网售票系统，使用 SpringCloud、Kafka、Redis、Sentinel、ElasticSearch、ShardingSphere 等，通过优化锁的策略、多级缓存管理、数据提前预热、精确的定制限流等多种策略，显著的降低下单延迟。此外还解决各种高并发难题的实际落地解决方案。是面试、就业、提高技术的不二选择！
confluentinc / librdkafka
The Apache Kafka C/C++ library
summer-rs / summer-rs
🍃spring-rs is a application framework written in rust inspired by java's spring-boot
86dbs / dbsyncer
DBSyncer（简称dbs）是一款开源的数据同步中间件，提供MySQL、Oracle、SqlServer、PostgreSQL、Elasticsearch(ES)、Kafka、File、SQL等同步场景。支持上传插件自定义同步转换业务，提供监控全量和增量数据统计图、应用性能预警等。
thingsboard / tbmq
The ultimate distributed MQTT broker. Handles 100M+ connections and 10M msg/sec with ease. Built on Kafka to provide industrial-grade persistence and eliminate data loss.
KouShenhai / KCloud-Platform-IoT
KCloud-Platform-IoT（老寇IoT云平台）是一个企业级单体架构和微服务架构的IoT云平台。采用DDD(领域驱动设计)思想，基于Spring Boot 4.1.1、Spring Cloud 2025.1.3、Spring Cloud Alibaba 2025.1.0.0 最新版本开发的云服务多租户IoT平台，家人们，点个star！拜托啦~
Aiven-Open / karapace
Karapace - Your Apache Kafka® essentials in one tool
linkedin / venice
Venice, Derived Data Platform for Planet-Scale Workloads.
apecloud / ape-dts
ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.
dapr / components-contrib
Community driven, reusable components for distributed apps
xxddpac / netra
One binary. Lightweight, high-performance. Observe your network with eBPF. Ask AI. Extend with MCP
vfaner / jync
Jync 捷同 - 极简、极速的异构数据库实时同步工具。单 JAR 部署，无需 Kafka/Zookeeper，零外部依赖。支持 MySQL、Oracle、PostgreSQL 及众多信创数据库。内置 DDL 自动同步、幂等 upsert、断点续传，内网离线可用。沐编程 (QQMu) 出品 | 官网: https://jync.qqmu.com