NVIDIA-NeMo / Curator

git clone https://github.com/NVIDIA-NeMo/Curator.git

Scalable data pre processing and curation toolkit for LLMs

—

stars, 24h

—

stars, 7 days

—

stars, 30 days

1.8k

★ total

351

forks · 297 open issues

Star history starts 2026-10-20; the chart appears after the second daily snapshot.

LanguagePython

LicenseApache-2.0

Created2024-03-14 (2y ago)

Last pushtoday

OwnerOrganization

GitHub topics

data data-curation data-prep data-preparation data-processing data-processing-pipelines data-quality datacuration datarecipes deduplication fast-data-processing fine-tuning large-language-models large-scale-data-processing llm llm-data-quality llmapps python semantic-deduplication

Links

GitHub: github.com/NVIDIA-NeMo/Curator
Releases · Issues · Hacker News mentions