sparklyr / sparklyr

git clone https://github.com/sparklyr/sparklyr.git v.1.9.6 · 1mo ago

R interface for Apache Spark that connects to Spark clusters and lets you use dplyr, SQL, and ML pipelines from R.

Why you might care

Lets R users run dplyr filters, aggregations, SQL, and Spark ML pipelines against local or remote Spark clusters, including YARN, Kubernetes, Mesos, and Livy. Useful for R analysts who need to work with data too large for memory and want to stay in R syntax.

Alternatives

SparkRPySparkApache Sparkdbplyr

stars, 24h · #53,586

+-1

stars, 7 days · #71,859

+~-2

stars, 30 days · #71,645

971

★ total · -0% this week

311

forks · 351 open issues

Star history (as recorded here)

2026-10-20 · 972 ★ 2026-11-06 · 971 ★

LanguageR

LicenseApache-2.0

Created2016-05-20 (10y ago)

Last push1mo ago

Watchers66

OwnerOrganization

Seen on GitHub Trending

2026-10-31 — #15 daily in r

GitHub topics

r rstats apache-spark machine-learning dplyr sparklyr remote-clusters livy ide distributed spark

README

Loading README…

Blurb generated by claude-haiku-5-5 6d ago. Star deltas marked ~ are extrapolated from partial history.