Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

markdown crawler scraper ai html-to-markdown web-crawler scraping web-scraper web-scraping data-extraction webscraping web-data-extraction ai-agents web-search ai-search web-data llm ai-crawler ai-scraping

• Updated Nov 12, 2026
• TypeScript

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

python mcp web-scraping cdp browser-automation ai-agents web-automation headless-browser remote-browser playwright cloud-browser ai-tools browser-agent llm computer-use stealth-browser browser-use

• Updated Nov 12, 2026
• Python

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

python crawler data automation ai mcp scraping crawling web-scraper web-scraping selectors xpath data-extraction stealth webscraping crawling-python playwright web-scraping-python ai-scraping mcp-server

• Updated Nov 11, 2026
• Python

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

python markdown open-source crawler scraper ai mcp web-crawler web-scraping data-extraction web-crawling ai-agents rag playwright llm crawl4ai-cloud

• Updated Nov 10, 2026
• Python

Scrapy, a fast high-level web crawling & scraping framework for Python.

python crawler framework scraping crawling web-scraping hacktoberfest web-scraping-python

• Updated Nov 10, 2026
• Python

Clone any website with one command using AI coding agents

react template boilerplate automation typescript ai clone skills nextjs reverse-engineering web-scraping developer-tools ai-agents claude tailwindcss website-clone ai-tools shadcn-ui claude-code

• Updated Nov 10, 2026
• TypeScript

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!

notifications rss monitoring self-hosted web-scraping website-monitor url-monitor change-alert change-detection website-change-monitor website-change-tracker website-monitoring change-monitoring website-watcher website-change-detector restock-monitor website-change-detection website-change-notification back-in-stock website-defacement-monitoring

• Updated Nov 12, 2026
• Python

Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.

python recaptcha selenium fingerprint chromium cloudflare web-scraping webscraping browser-automation bot-detection ai-agents anti-detect undetected puppeteer captcha-bypass headless-browser cloudflare-bypass playwright antidetect-browser stealth-browser

• Updated Nov 11, 2026
• Python

Python scraper based on AI

markdown crawler web-crawler scraping web-scraper web-scraping data-extraction webscraping web-data-extraction web-search ai-search rag web-data scraping-python web-crawlers llm ai-crawler large-language-model ai-scraping firecrawl-alternative

• Updated Oct 6, 2026
• Python

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

nodejs javascript npm crawler scraper automation typescript web-crawler headless scraping crawling web-scraping web-crawling headless-chrome apify puppeteer playwright

• Updated Nov 12, 2026
• TypeScript

The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.

python typescript ai web-scraping data-extraction agents cdp browser-automation headless-chrome ai-agents web-automation playwright cloud-browser browser-agent

• Updated Nov 12, 2026
• TypeScript

Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

rust browser servo web-crawler headless web-scraper web-scraping web-crawling browser-automation ai-agents puppeteer headless-browser playwright cloud-browser ai-tools kitesurf

• Updated Nov 12, 2026
• Rust

🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.

crawler scraper downloader social-media rest-api openapi video-downloader web-scraping bilibili data-collection speech-to-text douyin tiktok fastapi tiktok-scraper tiktok-api douyin-api tiktok-downloader model-context-protocol mcp-server

• Updated Nov 9, 2026
• Python

Turn any website into a structured API. Extract, automate, search and monitor the web.

api crawler scraper automation crawling web-scraper self-hosted web-scraping data-extraction webscraping agents browser-automation no-code web-search rpa robotic-process-automation nocode playwright

• Updated Nov 9, 2026
• TypeScript

Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9

agent ai self-hosted web-scraping agents rag notebooklm self-hosted-ai notebooklm-alternative

• Updated Nov 12, 2026
• Python

Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection

github python pdf documentation automation ocr mcp code-analysis web-scraping ast-parser documentation-generator conflict-detection multi-source github-scraper ai-tools claude-ai mcp-server claude-skills

• Updated Sep 30, 2026
• Python

Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.

chrome-extension agent automation browser muse web-scraping dia cdp browser-automation ai-agents web-automation puppeteer n8n web-agent manus playwright ai-tools browser-agent browser-use chrome-agent

• Updated Nov 10, 2026
• TypeScript

Python APIs for web automation, testing, and bypassing bot-detection.

python mcp test-automation pytest web-scraping chromedriver end-to-end-testing cdp bot-detection web-automation chrome-devtools-protocol selenium-python e2e-testing cloudflare-bypass seleniumbase playwright

• Updated Nov 6, 2026
• Python

Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.

nodejs javascript automation scraping web-scraping anti-bot browser-automation bot-detection puppeteer headless-browser cloudflare-bypass playwright ai-agent antidetect-browser stealth-browser

• Updated Nov 10, 2026
• JavaScript

jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

css java html parser dom xml jsoup web-scraping xpath css-selectors java-html-parser

• Updated Nov 11, 2026
• Java