D4Vinci / Scrapling

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

seleniumbase / SeleniumBase

📊 Python APIs for Web Automation, E2E Testing, and Data Scraping. CDP Mode bypasses bot-detection and handles CAPTCHAs. Includes a Stealth mode for Playwright.

tinyfish-io / agentql

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.

scrapfly / scrapfly-scrapers

Scalable Python web scraping scripts for +40 popular domains

makralabs / makra

Makra is a memory layer between the web and your AI agents. It serves real-time structured data from the open web at the cost of a vector search.