kalil0321 / reverse-api-engineer
The agent that turns websites into APIs!
kalil0321 / reverse-api-engineer
D4Vinci / Scrapling
🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
browser-use / browser-use
Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.
unclecode / crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
vibheksoni / stealth-browser-mcp
Stealth browser automation for AI agents over MCP and the Chrome DevTools Protocol: navigation, network hooks, DOM extraction, and pixel-accurate UI cloning.
3441293738 / creatorhub
多平台内容监控·采集·搬运 一个 Web 面板管起抖音 / 小红书 / 快手
dgtlmoon / changedetection.io
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!
CloakHQ / CloakBrowser
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
MODSetter / SurfSense
Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9
Mahanaicoach / google-maps-scraper-kit
Free Google Maps scraper for lead generation. Exports phones, emails, websites and socials to CSV. Runs locally in Docker, no API keys. Claude Code skill included.
scrapy / scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
seleniumbase / SeleniumBase
📊 Python APIs for Web Automation, E2E Testing, and Data Scraping. CDP Mode bypasses bot-detection and handles CAPTCHAs. Includes a Stealth mode for Playwright.
apify / crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
vladkens / twscrape
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
Coding-Doctor-Omar / reddrix
Reddrix is an open-source, blazing-fast Reddit Scraper for Python (No Login or API Key)
yusufkaraaslan / Skill_Seekers
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
adbar / trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
lexiforest / curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
feder-cr / invisible_playwright
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
Kaliiiiiiiiii-Vinyzu / patchright-python
Undetected Python version of the Playwright testing and automation library.
Evil0ctal / Douyin_TikTok_Download_API
🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Altimis / Scweet
Scrape tweets, profiles, followers and following from Twitter/X, no API key needed. Python library with smart multi-account pooling, proxy support and async.
tinyfish-io / agentql
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
feder-cr / invisible_playwright_mcp
Playwright MCP server undetected by anti-bots and captchas: AI agent browses the web on anti-detect stealth Firefox, Python, undetected browser automation, scraping, computer use.
chienbm98 / foxprofile
Free, open-source anti-detect browser profile manager: per-profile fingerprint, cookies and proxy, Camoufox and Chrome engines, desktop app, REST API and MCP server for AI agents. Self-hosted alternative to GoLogin / MoreLogin.
snooppr / snoop
Snoop — инструмент разведки на основе открытых данных (OSINT world)
rushter / selectolax
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
0x676e67 / wreq-python
An ergonomic, privacy-aware Python HTTP Client
scrapfly / scrapfly-scrapers
Scalable Python web scraping scripts for +40 popular domains
Anil-matcha / awesome-agent-apis
660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.
ail-project / ail-framework
AIL framework - Analysis Information Leak framework
runesleo / x-reader
Universal content reader MCP Server for 10+ platforms
ttlns / Selenium-Driverless
a stealthy browser automation framework
feder-cr / invisible_selenium
Selenium that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
tiliondev / fortress
Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.
n1tr00-10 / tiktok-web-signature
tiktok web signature reverse engineer generates valid tiktok signatures for API requets
cporter202 / coreclaw-api-directory
An unofficial, categorized directory of 118 CoreClaw Worker APIs for web scraping, automation, lead generation, e-commerce, social data, and more.
yail259 / frankensurf
One call that gets your agent through any site, stitched from 28 browsing tools
RayenMlayeh / jevcrawl
MCP server and CLI that crawls websites for AI agents: finds the answer page 2x more often than Crawl4AI, 10x cheaper per answer than letting Claude crawl.
00200200 / maintainer-skills-lab
Curated developer tools, AST analyzers, and automated verification workflows.
nodemaven / google-browser-scraper
Self-hosted Google search scraper for Python. Scrape Google SERP results with a real Chrome browser and your own residential proxy. Self-hosted SERP API alternative with JSON output, an HTTP API and an MCP server for AI agents.
sami-mag07 / ultimate-scraper-agent
Research skill and researcher subagent for Claude Code: multi-engine search behind one interface, parallel agent waves, and a verify gate that blocks any citation without a verbatim quote. No personal data; configured via .env.
vincenzo-afk / AgentWeb
Free, dependency-light Internet intelligence MVP for grounded research, extraction, citations, and page monitoring.
shanliuling / WxArticleSaver
本地运行的微信公众号文章导出工具，一键保存为 Markdown / HTML / TXT，支持文章内图片与视频下载
kowalski76057 / cf-turnstile-token
Scriptable Cloudflare Turnstile token CLI via the Peak API - sitekey discovery, solve, and token output for CI and automation.
nodemaven / proxy-benchmark
Measurement harness for proxy providers, browser engines and scraping targets. Every claim carries the run it came from.
makralabs / makra
Makra is a memory layer between the web and your AI agents. It serves real-time structured data from the open web at the cost of a vector search.
Encephos / markdown-link-resolver
A zero-dependency Python tool to resolve relative Markdown links and inline images as Base64 strings for RAG and Multimodal LLMs.
maximilianfeix / proxy-scraper
Free HTTP/SOCKS proxies with every check on the record: passed, failed or couldn't run, and when. Hourly list, --pick --verify, --diagnose, rotating proxy, Python API, MCP server.
CircuitSavage / cloudflare-turnstile-reversed
Cloudflare Turnstile challenge internals: the live request flow, the challenge bundle, and a capture toolkit.
Maas6696 / cloudflare-challenge-clearance
Bypass the Cloudflare challenge (Just a moment / 5s) in Python and get a valid cf_clearance cookie via the Peak API - headless-friendly, proxy-aware.
n1tr00-10 / tiktok-msddk-info-fully-reversed
Tiktok x msddk info fully reverse enginereed comes with fully algoryhtim and decrypter.
n1tr00-10 / tiktok-signature
TikTok X-Gnarly, X-Bogus, and X-Dynosaur signature generator for Web API requests. Supports SDK version 5.1.2 (build 1.0.0.316). Python implementation with ChaCha20 and RC4 encryption, dynamic round selection, and custom Base64 encoding. Lightweight, fast, and production-ready.