Refine high-quality datasets and visual AI models
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
A light-weight, flexible, and expressive statistical data testing library
Machine learning with dataframes
A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
Schema-Inspector is a simple JavaScript object sanitization and validation module.
A Python/Pandas ETL pipeline that turns a deliberately messy automotive dataset into production-ready data, with 15 pytest tests and GitHub Actions CI.
DeepSeek Harness plugin: deterministic data profiling, cleaning, and verification (dsh-data-quality)
Automated data preparation for machine learning. Load a dataset, clean, transform, encode, scale and split it, explore it with built-in analysis, and export model-ready data with a full report. Built with Python and Streamlit.
Project 01: Data Cleaning on a Real E-commerce Dataset