microsoft / markitdown

Python tool for converting files and office documents to Markdown.

rocketride-org / rocketride-server

High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.

infiniflow / ragflow

RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.

opendatalab / MinerU

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

docling-project / docling

Get your documents ready for gen AI

xevrion / hatchzip

HatchZip is a free, open-source archive extractor for Windows, macOS and Linux that extracts ZIP, 7z, RAR, tar and split archives in place, so it needs free space for the extracted files only. Desktop app and CLI.

VectifyAI / PageIndex

📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG

Evil0ctal / Douyin_TikTok_Download_API

🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.

redis / redis

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

DataExpert-io / data-engineer-handbook

This is a repo with links to everything you'd ever want to learn about data engineering

DataTalksClub / data-engineering-zoomcamp

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

datalab-to / indra

1M param OCR model - runs on CPU, 15x faster than tesseract, close to 1B+ param models in accuracy

HKUDS / LightRAG

[EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation

FengZhenfei / carrel

Ontology-Augmented Generation for AI agents.

seaweedfs / seaweedfs

SeaweedFS is a distributed storage system for object storage (S3), file systems, Iceberg table buckets, and Lance vector buckets, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

datalab-to / marker

Convert PDF to markdown + JSON quickly with high accuracy

YOLOW92 / pagesift

Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.

opendataloader-project / opendataloader-pdf

PDF Parser for AI-ready data. Automate PDF accessibility. Open-source.

Unstructured-IO / unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

DataWithBaraa / claude-code-data-engineering-skills

16 Claude Code skills for data engineers: profile a new source, investigate a failed run, review a change, document the platform. Most end in a page you can share.

apache / doris

Apache Doris is a real-time analytics and hybrid search database for AI agents.

CAPCOM-TD-OSS / REDox

High-performance, token-based structured data engine for .NET. A core component of REX, the technology behind CAPCOM's next-generation game engine.

teslamate-org / teslamate

A self-hosted data logger for your Tesla 🚘 [main maintainer=@JakobLichterfeld]

colinhacks / motherload

Import data files from TypeScript, fully typed. Powered by TypeScript 7.1 content mappers.

datalab-to / lift

Extract structured data from documents quickly and accurately.

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

fluent / fluent-bit

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

apache / carbondata

High performance data store solution

stackabletech / docker-images

Product images for the Stackable Data Platform