All repos — GH Trending Explorer

Unified analytics engine for large-scale data processing with APIs in Scala, Java, Python, and R.

data-engineering data-science backend

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator

firesim / firesim

FireSim: Fast and Effortless FPGA-accelerated Hardware Simulation with On-Prem and Cloud Flexibility

Eezo-framework / eezo

A Scala 3 web framework. Direct-style. The case class is the source of truth. Deploy with one command.

spotify / scio

A Scala API for Apache Beam and Google Cloud Dataflow.

snowplow / snowplow

The leader in Customer Data Infrastructure

microsoft / SynapseML

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

zio / zio

ZIO — A type-safe, composable library for async and concurrent programming in Scala

awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

joernio / joern

Open-source code analysis platform for C/C++/Java/Binary/Javascript/Python/Kotlin based on code property graphs. Discord https://discord.gg/vv4MH284Hc

Netflix / atlas

In-memory dimensional time series database.

metarank / metarank

A low code Machine Learning personalized ranking service for articles, listings, search results, recommendations that boosts user engagement. A friendly Learn-to-Rank engine

apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

typelevel / doobie

Functional JDBC layer for Scala.

zio / zio-quill

Compile-time Language Integrated Queries for Scala

OpenBankProject / OBP-API

An open source RESTful API platform for banks that supports Open Banking, XS2A, PSD2 and Open Finance through access to accounts, transactions, counterparties, payments, entitlements and metadata - plus a host of internal banking and management APIs.

typelevel / skunk

A data access library for Scala + Postgres.

combust / mleap

MLeap: Deploy ML Pipelines to Production

locationtech / geomesa

GeoMesa is a suite of tools for working with big geo-spatial data in a distributed fashion.

filodb / FiloDB

Distributed Prometheus time series database

apache / carbondata

High performance data store solution

locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.

scalikejdbc / scalikejdbc

A tidy SQL-based DB access library for Scala developers. This library naturally wraps JDBC APIs and provides you easy-to-use APIs.

scalalandio / chimney

Scala library for boilerplate-free, type-safe data transformations

graphframes / graphframes

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

lensesio / stream-reactor

A collection of Kafka Connect sinks and sources maintained by Lenses.io.

airbnb / chronon

Chronon is a data platform for serving for AI/ML applications.

NVIDIA / cudf-spark

NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs

apache / livy

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

delta-io / delta-sharing

An open protocol for secure data sharing

dbpedia / extraction-framework

The software used to extract structured data from Wikipedia

RedisLabs / spark-redis

A connector for Spark that allows reading and writing to/from Redis cluster

ReactiveMongo / ReactiveMongo

:leaves: Non-blocking, Reactive MongoDB Driver for Scala

LucaCanali / sparkMeasure

This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simplifies collecting, aggregating, and exporting Spark task/stage metrics, and is designed for practical use by developers and data engineers in interactive analysis, testing, and production monitoring workflows.