📈 GH Trending Explorer 🎲 Surprise me

← all repos

D4Vinci / Scrapling

git clone https://github.com/D4Vinci/Scrapling.git v0.5 · 5d ago

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

+53
stars, 24h · #280
+~371
stars, 7 days · #341
+~1.6k
stars, 30 days · #339
89k
★ total · +0% this week
9.2k
forks · 15 open issues

Star history (as recorded here)

2026-10-20 · 88,864 ★ 2026-10-22 · 89,080 ★
LanguagePython
LicenseBSD-3-Clause
Created2024-10-13 (2y ago)
Last push1d ago
Watchers324
OwnerUser

GitHub topics

Links

README excerpt
Effortless Web Scraping for the Modern Web 
 Selection methods 
 ·
 Fetchers 
 ·
 Spiders 
 ·
 Proxy Rotation 
 ·
 CLI 
 ·
 MCP 
Scrapling is a Python Web Scraping framework that brings fast HTTP requests, stealth browsers, powerful parsing, and concurrent crawling into one library.
Scrape static pages, render JavaScript, interact with websites, and extract data from the API calls behind them. Build crawlers with reusable sessions, proxy rotation, automatic retries, pause/resume, and built-in data exports.
Use it from Python, the terminal, or your AI agent through its MCP server and Agent Skill. From extracting a few fields to collecting a whole website as clean Markdown, Scrapling covers the full scraping workflow.
## What you can do with Scrapling
- **Fetch & interact**: Fast HTTP requests, stealth browsers, JavaScript rendering, Cloudflare Turnstile/Interstitial handling, persistent sessions, and background API capture.
- **Parse & extract**: CSS, XPath, text and regex search, adaptive element tracking, similar-element matching, and clean Markdown output.
- **Crawl & export**: Concurrent spiders, proxy rotation, retries, AutoThrottle, pause/resume, streaming, ready-made templates, and JSON/JSONL/CSV/XML exports.
- **Scrape with AI or from your terminal**: An MCP server for AI agents to browse and extract data, an Agent Skill, CLI commands, and an interactive shell.
Start with a single page:
 [code] 
Or scale up with a spider:
 [code] 
This prints `10` and saves the quotes and authors to `quotes.json`.
# Platinum Sponsors
 NodeMaven - The most efficient proxy provider for Web Scraping and Automation with the Highest Quality IP on the market. Use code SCRAPLING35 for 35% discount.
 Scrapling handles Cloudflare Turnstile. For enterprise-grade protection, 
 Hyper Solutions 
 provides API endpoints that generate valid antibot tokens for Akamai , DataDome , Kasada , and Incapsula . Simple API calls, no browser automation required. 
 Evomi 
 : residential proxies from $0.49/GB. Scraping browser with fully spoofed Chromium, residential IPs, auto CAPTCHA solving, and anti-bot bypass. 
 Scraper API for hassle-free results. MCP and N8N integrations are available. 
 TikHub.io provides 900+ stable APIs across 16+ platforms including TikTok, X, YouTube & Instagram, with 40M+ datasets. Also offers DISCOUNTED AI models - Claude, GPT, GEMINI & more up to 71% off.
 Read a full review of Scrapling on The Web Scraping Club (Nov 2025), the #1 newsletter dedicated to Web Scraping.
 Swiftproxy provides scalable residential proxies with 80M+ IPs across 195+ countries, delivering fast, reliable connections, automatic rotation, and strong anti-block performance. Free trial available.
 Byteful supplies ethical residential, mobile, ISP, and datacenter proxies, plus scraping and AI automation tools. Use code SCRAPLING10: 10% off residential proxies.
 ProxyLane : Reduce blocks and CAPTCHAs by 90%. 28M Premium Residential IPs for Heavy Web Scraping. 195 countries. City/ISP/