jiji262 / douyin-downloader

A practical Douyin downloader for both single-item and profile batch downloads, with progress display, retries, SQLite deduplication, and browser fallback support. 抖音批量下载工具，去水印，支持视频、图集、合集、音乐(原声)。

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.

moj-analytical-services / splink

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

NVIDIA-NeMo / Curator

Scalable data pre processing and curation toolkit for LLMs

RazgrizHsu / immich-deduper

duplicate photo finder for Immich - find and remove similar images

toki-plus / video-mover

全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

karanhudia / borg-ui

Replace complex Borg Backup terminal commands with a beautiful web UI. Create, schedule, and restore backups with just a few clicks.

ayushjain1144 / trackeverything

Code repository for "TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations"

bup / bup

Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images). Please post problems or patches to the mailing list for discussion (see the end of the README below).

borgmatic-collective / borgmatic

Mirror of borgmatic: Simple, configuration-driven backup software for servers and workstations

agentic-box / memora

Give your AI agents persistent, collective memory — with deduplicating absorb, supersession lineage, semantic search, and a graph UI. Speaks MCP.

MinishLab / semhash

Fast Multimodal Semantic Deduplication & Filtering

lincwang123-bot / humanized-social-publisher

Agent-friendly local publisher for Xiaohongshu, Douyin and WeChat Channels with multi-account safety, scheduling and deduplication.

researchradar / research-radar

Self-hosted research intelligence for collecting, deduplicating, ranking, searching, and browsing the researchers, topics, and sources you care about.

ra1nty / DXcam

A Python high-performance screen capture library for Windows using Desktop Duplication API - Updated 2026

ivandilelio / kybes

Know Your Backend is a Plugin that help your AI to generate a structured Model/View/Controller application, witouth surprise, code duplication and different modality in a different project