apache / datafusion-comet
git clone https://github.com/apache/datafusion-comet.git
1.1.0 · 18d ago
🔥 on GitHub Trending today
Apache DataFusion Comet is a plugin that accelerates Apache Spark queries by running operators and expressions natively on Arrow columnar data.
Why you might care
Useful for teams running Spark SQL, DataFrame, or PySpark jobs who want lower runtime and cloud cost without rewriting code. It executes operators in Rust via DataFusion or on the JVM over Arrow batches, and requires no GPUs or specialized hardware. The README reports about a 1.8x speedup on TPC-DS SF1000 versus Spark 4.2, though that is the project's own benchmark.
Star history (as recorded here)
Seen on GitHub Trending
- 2026-10-24 — #22 daily in scala · +1 ★
- 2026-10-24 — #6 monthly in scala · +19 ★
- 2026-10-24 — #4 weekly in scala · +8 ★
- 2026-10-23 — #6 monthly in scala · +20 ★
- 2026-10-23 — #4 weekly in scala · +7 ★
- 2026-10-22 — #6 monthly in scala · +20 ★
- 2026-10-22 — #4 weekly in scala · +9 ★
- 2026-10-21 — #11 daily in scala · +1 ★
- 2026-10-21 — #5 monthly in scala · +20 ★
- 2026-10-21 — #3 weekly in scala · +8 ★
- 2026-10-20 — #11 daily in scala · +1 ★
- 2026-10-20 — #5 monthly in scala · +19 ★
- 2026-10-20 — #3 weekly in scala · +8 ★
GitHub topics
Links
- GitHub: github.com/apache/datafusion-comet
- Homepage: https://datafusion.apache.org/comet
- Releases · Issues · Hacker News mentions
README
Loading README…
Blurb generated by claude-haiku-5-5 1d ago. Star deltas marked ~ are extrapolated from partial history.