Ephemeral-AI-Lab / layerfs
Ephemeral Workspaces, durable shared history. LayerFS gives every agent a complete, writable filesystem Workspace over shared committed history, without copying the base. Agents work in parallel, commit what changed as deduplicated checkpoints, then branch, rewind or merge. Try it with LayerFS × DSH.
surya-koritala / Glyd
Lossless AI compression: 33% less GPU memory, bit for bit.
StoneSniperStrip / Discord-Server-Cloner
Discord Server Cloner 2025: Full working installer with all features enabled. Ready-to-run desktop build for easy server duplication.
restic / restic
Fast, secure, efficient backup program
jiji262 / douyin-downloader
A practical Douyin downloader for both single-item and profile batch downloads, with progress display, retries, SQLite deduplication, and browser fallback support. 抖音批量下载工具，去水印，支持视频、图集、合集、音乐(原声)。
kopia / kopia
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
kucherenko / jscpd
Copy/paste detector for source code. 220+ languages, Rust engine, SARIF/HTML/badge reporters, GitHub Action, MCP server for AI agents.
tool-bunker / code-buster
Code Buster reports dependency wiring, dead code, cycles, duplication, complexity, architecture-policy violations, feature flags, security and style heuristics, quality scores, and remediation plans.
fallow-rs / fallow
Codebase intelligence for TypeScript and JavaScript. Health, complexity hotspots, duplication, architecture boundaries, circular dependencies, design-system drift, and unused code, from one graph. CLI, GitHub Action, LSP, MCP, and VS Code. Rust, MIT licensed.
scanaislop / aislop
Catch and fix the code-quality issues AI coding agents leave behind - dead code, unsafe casts, swallowed errors, duplication, security risks, and more. 50+ deterministic rules across 10 language targets, with CLI, CI, and GitHub Actions. No LLM at runtime. MIT.
toki-plus / video-mover
全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.
borgbackup / borg
Deduplicating archiver with compression and authenticated encryption.
windirstat / windirstat
Windows disk usage analyzer and cleanup tool with interactive treemaps, large-file discovery, duplicate detection, and file-system monitoring.
moj-analytical-services / splink
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
PlakarKorp / plakar
plakar is a backup solution powered by Kloset and ptar
poncippg-spec / liquidDX11
Real-time desktop glass for Dear ImGui using Win32, DirectX 11, DXGI Desktop Duplication, and HLSL.
prometheus / alertmanager
Prometheus Alertmanager
bup / bup
Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images). Please post problems or patches to the mailing list for discussion (see the end of the README below).
mhx / dwarfs
A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows
OpenGene / fastp
An ultra-fast all-in-one FASTQ preprocessor (QC/adapters/trimming/filtering/splitting/merging...)
sahib / rmlint
Extremely fast tool to remove duplicates and other lint from your filesystem
borgmatic-collective / borgmatic
Mirror of borgmatic: Simple, configuration-driven backup software for servers and workstations
NVIDIA-NeMo / Curator
Scalable data pre processing and curation toolkit for LLMs
karanhudia / borg-ui
Replace complex Borg Backup terminal commands with a beautiful web UI. Create, schedule, and restore backups with just a few clicks.
zinggAI / zingg
Scalable master data management, identity resolution, entity resolution, and deduplication using ML
darula-hpp / shimmer-from-structure
A structure-aware skeleton loader that mirrors your rendered UI at runtime. Zero layout duplication. Built for modern frameworks.
MinishLab / semhash
Fast Multimodal Semantic Deduplication & Filtering
data-prep-kit / data-prep-kit
Open source project for data preparation for GenAI applications
ra1nty / DXcam
A Python high-performance screen capture library for Windows using Desktop Duplication API - Updated 2026
rin-nas / postgresql-patterns-library
Коллекция готовых SQL запросов для PostgreSQL по часто возникающим задачам (получение и модификация данных, ускорение запросов, обслуживание БД)
RazgrizHsu / immich-deduper
duplicate photo finder for Immich - find and remove similar images
CarlosZ16420 / hamster-archiver
Windows local-first file organizer and media catalog with MCP tools for AI assistants. Verified batch archiving. 本地文件整理、媒体建库与可校验批量归档，支持 AI 通过 MCP 调用。
ivandilelio / kybes
Know Your Backend is a Plugin that help your AI to generate a structured Model/View/Controller application, witouth surprise, code duplication and different modality in a different project
alt8cio-source / tarkov-stash-replication-utility
Tarkov Item Duplication Exploit 2026: Dupe Any Item in 30 Seconds with FIR Status
elliribeiro1-spec / tarkov-fir-stash-mirror
Escape from Tarkov Item Duplication Guide 2026: Stash Dupe Method for All Items
tools4imps / exhale-ruby
The contraction gate for Rails: no PR merges while the codebase holds duplication the Contract doesn't keep
lincwang123-bot / humanized-social-publisher
Agent-friendly local publisher for Xiaohongshu, Douyin and WeChat Channels with multi-account safety, scheduling and deduplication.
Aaqibhafeezkhan / OpenNewsGrid
Real-time global news aggregation platform built with Next.js 15, TypeScript, and Tailwind CSS. It aggregates news from 300+ international sources through RSS feeds, NewsAPI, GNews, and other APIs, with smart deduplication to prevent duplicate stories.