h4ckf0r0day / obscura

The headless browser for AI agents and web scraping

lexmount / moli

Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

spider-rs / spider

Foundational low latency web data collecting in Rust

0xMassi / webclaw

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

plabayo / rama

modular service framework to move and transform network packets

neon-mmd / websurfx

:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine

austin-weeks / miasma

Trap AI web scrapers in an endless poison pit.

fastcrw / crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

janreges / siteone-crawler

SiteOne Crawler is a cross-platform website crawler and analyzer for SEO, security, accessibility, and performance optimization—ideal for developers, DevOps, QA engineers, and consultants. Supports Windows, macOS, and Linux (x64 and arm64).

proofofbots / web-re-toolkit

Headless Akamai & Kasada solver that runs without a browser, and the reversing toolkit it was built with