apache / spark
Apache Spark - A unified analytics engine for large-scale data processing
apache / spark
delta-io / delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
apache / couchdb
Seamless multi-primary syncing database with an intuitive HTTP/JSON API, designed for reliability
ClickHouse / ClickHouse
ClickHouseยฎ is a real-time analytics database management system
apache / flink
Apache Flink
thingsboard / thingsboard
All-in-one IoT Platform - Device management, data collection, processing and visualization.
amark / gun
An open source cybersecurity protocol for syncing decentralized graph data.
prestodb / presto
The official home of the Presto distributed SQL query engine for big data
andkret / Cookbook
The Data Engineering Cookbook
trinodb / trino
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
StarRocks / starrocks
The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
quickwit-oss / quickwit
Cloud-native OSS search engine for observability
cython / cython
The most widely used Python to C compiler
apache / datafusion
Apache DataFusion SQL Query Engine
catboost / catboost
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
apache / beam
Apache Beam is a unified programming model for Batch and Streaming data processing.
arkime / arkime
Arkime is an open source, large scale, full packet capturing, indexing, and database system.
h2oai / h2o-3
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
feast-dev / feast
The Open Source Feature Store for AI/ML
vespa-engine / vespa
The AI search platform
apache / zeppelin
Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.
hazelcast / hazelcast
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
apache / iotdb
Apache IoTDB
apache / hive
Apache Hive
Eventual-Inc / Daft
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
microsoft / SynapseML
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
apache / calcite
Apache Calcite
apache / ignite
Apache Ignite
crate / crate
CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.
alibaba / fastjson2
๐ FASTJSON2 is a Java JSON library with excellent performance.
alibaba / GraphScope
๐จ ๐ ๐ป ๐ GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | ไธ็ซๅผๅพ่ฎก็ฎ็ณป็ป
apache / datafusion-sqlparser-rs
Extensible SQL Lexer and Parser for Rust
lakehq / sail
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
apache / paimon
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
apache / hugegraph
A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
TuiQiao / CBoard
An easy to use, self-service open BI reporting and BI dashboard platform.
kafbat / kafka-ui
Open-Source Web UI for managing Apache Kafka clusters
man-group / ArcticDB
ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
quarylabs / quary
Open-source BI for engineers
ytsaurus / ytsaurus
YTsaurus is a scalable and fault-tolerant open-source big data platform.
apache / ambari
Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.
apache / fluss
Apache Fluss is a streaming storage built for real-time analytics.
apache / datafusion-ballista
Apache DataFusion Ballista Distributed Query Engine
apache / drill
Apache Drill is a distributed MPP query layer for self describing data
apache / bookkeeper
Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads
fluid-cloudnative / fluid
Fluid, elastic data abstraction and acceleration for BigData/AI applications in cloud. (Project under CNCF)
apache / kudu
Mirror of Apache Kudu
apache / auron
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Netflix / genie
Distributed Big Data Orchestration Service
apache / carbondata
High performance data store solution
apache / cloudberry
One advanced and mature open-source MPP (Massively Parallel Processing) database. Open source alternative to Greenplum Database.
uxlfoundation / scikit-learn-intelex
Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application
apache / ozone
Scalable, reliable, distributed storage system optimized for data analytics and object store workloads.
mukunku / ParquetViewer
Simple Windows desktop application for viewing & querying Apache Parquet files
graphframes / graphframes
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
apache / amoro
Apache Amoro(incubating) is a Lakehouse management system built on open data lake formats.
apache / accumulo
Apache Accumulo
ClickHouse / ClickBench
ClickBench: a Benchmark For Analytical Databases
metabrainz / listenbrainz-server
Server for the ListenBrainz project, including the front-end (javascript/react) code that it serves and all of the data processing components that LB uses.
apache / flink-kubernetes-operator
Apache Flink Kubernetes Operator