wistbean / learn_python3_spider
python爬虫教程系列、从0到1学习python爬虫，包括浏览器抓包，手机APP抓包，如 fiddler、mitmproxy，各种爬虫涉及的模块的使用，如：requests、beautifulSoup、selenium、appium、scrapy等，以及IP代理，验证码识别，Mysql，MongoDB数据库的python使用，多线程多进程爬虫的使用，css 爬虫加密逆向破解，JS爬虫逆向，分布式爬虫，爬虫项目实战实例等
s0md3v / Photon
Incredibly fast crawler designed for OSINT.
cv-cat / Spider_XHS
小红书爬虫数据采集，小红书逆向，私信，直播，小红书全域运营解决方案
DedSecInside / TorBot
Dark Web OSINT Tool
cv-cat / DouYin_Spider
抖音逆向，抖音爬虫，抖音全部api、私信、直播间监听
scrapy / scrapyd
A service daemon to run Scrapy spiders
howie6879 / owllook
owllook-小说搜索引擎
amirh00sain / SpiderPanel
—
howie6879 / ruia
Async Python 3.6+ web scraping micro-framework based on asyncio
eatmoreduck / boss-zhipin-scraper
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BOSS Zhipin job scraper/crawler.
blacklanternsecurity / MANSPIDER
Spider entire networks for juicy files sitting on SMB shares. Search filenames or file content - regex supported!
okfn-brasil / querido-diario
📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian government gazettes, accessible to everyone.
scrapfly / scrapfly-scrapers
Scalable Python web scraping scripts for +40 popular domains
jasoneri / ComicGUISpider
漫画下载软件：拷贝漫画, Māngabz, jestful, 漫画柜, 动漫屋, 禁漫天堂, wnacg, exhentai, nhentai, hitomi.la, h-comic ; 附有 kemono, danbooru 等衍生下载工具
0xAllenChen / spider_reverse
爬虫逆向案例，已完成：TLS指纹｜瑞数｜震坤行 | 网易易盾 | 微信小程序反编译逆向(百达星系) | 同花顺 | rpc解密 | 加速乐 | 极验滑块验证码 | 巨量算数 | Boss直聘 | 企查查 | 中国五矿 | qq音乐 | 产业政策大数据平台 | 企知道 | 雪球网(acw_sc__v2) | 1688 | 七麦数据 | whggzy | 企名科技 | mohurd | 艺恩数据 | 欧科云链
scrapinghub / scrapyrt
HTTP API for Scrapy spiders
alltheplaces / alltheplaces
A set of spiders and scrapers to extract location information from places that post their location on the internet.
BlessedRebuS / Krawl
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities using realistic, randomly generated decoy data and AI-generated HTML templates.
polyrabbit / hacker-news-digest
:newspaper: Let LLM Summarize Hacker News for You
yeximm / Access_wechat_article
微信文章爬虫，用于采集、整理和归档微信公众号公开文章元数据，欢迎一起学习讨论。A Windows research workbench for collecting, organizing, and archiving public WeChat article metadata.
facebookresearch / spider
A general physic-based retargeting framework.
CharlesPikachu / freeproxy
FreeProxy: Collecting free proxies from internet. (全球海量高质量免费代理，支持爬取数十个免费代理分享源，支持自定义规则代理筛选，爬虫与数据分析必备，每日更新海量免费代理。)
scrapinghub / spidermon
Scrapy Extension for monitoring spiders execution.
aoyunyang / spider-king-skill
Protocol-first reverse engineering skill for turning hostile web clients into pure-protocol Python collectors.
h100envy / gem-search
Rainbow spider that X-rays memecoins: bundles, linked wallets, dev history. Scanner, Telegram bot, live Bundle Index.
cv-cat / KuaiShou-Spider
快手逆向，快手爬虫，快手全部api、直播间监听
Ouru77 / ev-assistant
🕸️ E.V. — Peter Parker's homemade AI from Spider-Man: Brand New Day, running locally on YOUR PC. Voice assistant + cyber HUD, real PC control, long-term memory. Whisper + Ollama, no cloud. TR/EN.
iltaen / spider_walker
Blender addon: Procedural walking for spider-like armatures