Learn System Design concepts and prepare for interviews using free resources.
For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
Distributed reliable key-value store for the most critical data of a distributed system
Learn how to design systems at scale and prepare for system design interviews
The java implementation of Apache Dubbo. An RPC and microservice framework.
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.
SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.
Conductor is an event driven agentic workflow engine providing durable and highly resilient execution engine for applications and AI Agents
If you want to become good at AI engineering & system design, join this newsletter 👇
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
Dapr is a portable runtime for building distributed applications across cloud and edge, combining event-driven architecture with workflow orchestration.
Temporal service
《Designing Data-Intensive Application》DDIA 第一版 / 第二版 中文翻译
High-Performance server for NATS.io, the cloud and edge native messaging system.
The lightweight, fault-tolerant database built on SQLite. Designed to keep your data highly available with minimal effort.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
A platform to build and run apps that are elastic, agile, and resilient. SDK, libraries, and hosted environments.
Apache ZooKeeper
Infrastructure orchestration from local to your cloud
Cloud Native application framework for .NET
Distributed, offline-first bug tracker integrated in git
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
Provides a simple way to run Selenium Grid with Chrome, Firefox, and Edge using Container Platform, making it easier to perform browser automation at scale
🪓 An orchestration engine for background tasks, AI agents, and durable workflows
Cloud-Native distributed storage built on and for Kubernetes
Broadcast, Presence, and Postgres Changes via WebSockets
Open Source, Google Zanzibar-inspired database for scalably storing and querying fine-grained authorization data
《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
Instant Kubernetes-Native Application Observability
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
A Cloud Native traffic orchestration system. (CNCF Project)
⟁ Tendermint Core (BFT Consensus) in Go
Modular, contributor-friendly and blazing-fast implementation of the Ethereum protocol, in Rust
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
Gluster Filesystem : Build your distributed storage in minutes
Dkron - Distributed, fault tolerant job scheduling system https://dkron.io
An actor-based Framework with network transparency for creating event-driven architecture in Golang. Inspired by Erlang. Zero dependencies.
Restate is the platform for building resilient applications that tolerate all infrastructure faults w/o the need for a PhD.
Next generation distributed, event-driven, parallel config management!
Symbolic Learning in Python and Julia
A list of useful resources to learn Data Engineering from scratch
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.
Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.
A modern high-performance open source message queuing system
Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale.
Documentation tool built for software architecture. Document your domains, services, events and schemas — for your teams and your AI agents.
The most complete DDS - Proven: Plenty of success cases. Looking for commercial support? Contact info@eprosima.com
Kafka-style message streaming in Go. Built on NATS. Single binary, no JVM, no ZooKeeper.
Modern applications with built-in automation
Vitess for Postgres
IoT Platform Framework
🧭 Architecture-first system design: 26 bilingual tutorials, 25 architecture templates, and 6 end-to-end cases covering distributed systems, AI-native systems, RAG, coding Agents, and production trade-offs.
The gold standard for async .NET microservices on Azure, AWS and on-prem