lance-format / lance

Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Eventual-Inc / Daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

apache / arrow-rs

Official Rust implementation of Apache Arrow

lakehq / sail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

shshemi / tabiew

A lightweight TUI application to view and query tabular data files, such as CSV, TSV, and parquet.

parseablehq / parseable

Parseable is an open source, unified infrastructure observability platform built in Rust on a data lake architecture. It tracks logs, metrics, traces, and events across apps, agents, and systems, reducing storage costs by up to 90% through columnar telemetry compression.

nisshi-io / nisshi

Apache Kafka® compatible broker with S3, PostgreSQL, SQLite, Apache Iceberg and Delta Lake

paradigmxyz / cryo

cryo is the easiest way to extract blockchain data to parquet, csv, json, or python dataframes

CrunchyData / pg_parquet

Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL

kylebarron / parquet-wasm

Rust-based WebAssembly bindings to read and write Apache Parquet data

Koukyosyumei / h5i-db

An agent-native workspace for quantitative research: an in-terminal notebook, a high-performance time-series database, and a high-throughput backtesting engine. Written in Rust.