apache / iggy

Website | Getting started | Documentation | Blog | Discord | Crates | Mailing list
**Contents:** What is Apache Iggy? ·
Features ·
Architecture ·
Clustering ·
Version ·
Supported languages SDK ·
CLI ·
Web UI ·
Connectors ·
Kafka gateway ·
Model Context Protocol ·
Docker ·
Kubernetes ·
Configuration ·
Run the server ·
Try the CLI ·
Examples ·
SDK ·
Benchmarks ·
Contributing
---
## What is Apache Iggy?
**Iggy** is a persistent message streaming platform written in Rust, supporting QUIC, WebSocket, TCP (custom binary specification) and HTTP (regular REST API) transport protocols, **capable of processing millions of messages per second at ultra-low latency**.
Iggy provides **exceptionally high throughput and performance** while utilizing minimal computing resources.
This is **not yet another extension** running on top of existing infrastructure, such as Kafka or SQL database.
Iggy is a persistent message streaming log **built from the ground up** using low-level I/O with **thread-per-core shared nothing architecture**, `io_uring` and `compio` for maximum speed and efficiency.
The name is an abbreviation for the Italian Greyhound - small yet extremely fast dogs, the best in their class. See the lovely Fabio & Cookie ❤️
---
## Features
- **Highly performant**, persistent append-only log for message streaming
- **Very high throughput** for both writes and reads
- **Low latency and predictable resource usage** thanks to the Rust compiled language (no GC) and `io_uring`.
- **User authentication and authorization** with granular permissions and Personal Access Tokens (PAT)
- Support for multiple streams, topics and partitions
- Support for **multiple transport protocols** (QUIC, WebSocket, TCP, HTTP)
- Fully operational RESTful API which can be optionally enabled
- Available client SDK in multiple languages
- **Thread per core shared nothing design** together with `io_uring` guarantee the best possible performance on modern `Linux` systems.
- **Works directly with binary data**, avoiding enforced schema and serialization/deserialization overhead
- Custom **zero-copy (de)serialization**, which greatly improves the performance and reduces memory usage.
- Configurable server features (e.g. caching and transport protocols), plus per-topic segment size, durability and flush thresholds
- Server-side storage of **consumer offsets**
- Multiple ways of polling the messages:
 - By offset (using the indexes)
 - By timestamp (using the time indexes)
 - First/Last N messages
 - Next N messages for the specific consumer
- Optional **poll auto-commit**; processing guarantees depend on application processing and offset-commit ordering
- **Consumer groups** providing the message ordering and horizontal scaling across the connected clients
- **Message expiry** with auto deletion based on the configurable **retention policy**
- Additional features such as **server side message deduplication**
- **Multi-tenant** support via abstraction of **streams** which group **topics**
-