Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.

go golang crawler cloning website-scraper website-cloner

• Updated Jun 2, 2026
• Go

Download website to local directory (including all css, images, js, etc.)

nodejs javascript scraper hacktoberfest website-scraper

• Updated Nov 11, 2026
• JavaScript

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.

python osint selenium tor information-extraction websites web-scraping webcrawler webscraping information-gathering website-scraper reconnaissance darkweb selenium-webscraper osint-python webcra osint-tool darkweb-crawler

• Updated Nov 24, 2024
• Python

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

nodejs json crawler scraper spider linkedin scraping crawling expressjs linkedin-profile scrapers scraping-websites linkedin-bot website-scraper profile-data linkedin-scraper linkedin-crawler puppeteer linkedin-scraping linkedin-profile-scraper

• Updated Apr 5, 2024
• TypeScript

Plugin for website-scraper which returns html for dynamic websites using puppeteer

nodejs javascript chrome scraper chromium hacktoberfest website-scraper puppeteer

• Updated Nov 11, 2026
• JavaScript

High-performance platform for building websites, e-commerce, and web applications—with Native AI, JavaScript development, and a marketplace for portable sites and apps.

javascript wordpress cms development ai templates magento website-builder shopify website-scraper kooboo ai-agents website-development ai-tools ai-agent web-application-platform ai-coding

• Updated Oct 14, 2026
• JavaScript

🕸 Turn almost any website into an RSS feed and serve it instantly. Automatic scraping. Start fast with Docker.

ruby docker rss scraper builder feed roda rss-feed rss-aggregator serves rss-feed-scraper website-scraper webfeeds webfeed html2rss rolling-release html2rss-configs feed-configs

• Updated Nov 11, 2026
• Ruby

DPULSE - Tool for complex approach to domain OSINT

intelligence osint web-scraping cybersecurity pentesting pentest information-security data-gathering webscraping information-gathering website-scraper intelligence-gathering pentest-tool domain-analysis google-dorking osint-tool cybersecurity-education osint-tools infosectools cybersecurity-tool

• Updated Sep 28, 2026
• Python

Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook.

crawler scraper web-crawler scraping web-scraper web-scraping data-extraction webscraping web-crawling data-scraping website-scraper web-data-extraction email-extractor page-scraper scraping-tool scrape-data scrape-products serp-scraper serp-scraping phone-number-extractor

• Updated Mar 17, 2026
• JavaScript

🌱 goClone - clone websites in seconds

go golang crawler cloner scraping crawling cloning scrapper scraping-websites hacktoberfest website-scraper website-cloner wp2static goclone

• Updated Nov 9, 2026
• Go

Wayback Machine Downloader. 🔥 Download your entire archived websites from the Internet Archive Wayback Machine.

console command-line-app csharp internet internet-archive command-line-tool console-application wayback-machine command-line-parser website-scraper console-app internet-wayback-machine wayback-machine-downloader

• Updated Aug 5, 2022
• C#

Automatically curates and posts content to LinkedIn. It can optionally use web scraping to gather data, which is then fed to ChatGPT to craft engaging LinkedIn posts.

linkedin linkedin-bot website-scraper linkedin-scraper post-scheduler openai-api post-scheduling linkedin-post chatgpt-api chatgpt-bot linkedin-posts-automation

• Updated Apr 13, 2025
• Python

Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.

crawler scraper user-agent scraping beautiful-soup robots-txt beautifulsoup scrapper website-scraper scrapping-python website-crawler beautifulsoup4 crawling-python iprotation

• Updated May 20, 2026
• Python

A server to collect & archive websites that also supports video downloads

self-hosted video-downloader archive webinterface web-archiving website-scraper website-archive

• Updated Feb 11, 2023
• TypeScript

ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.

crawler scraper telegram-bot proxy robots-txt website-scraper rag huggingface large-language-models llm retrieval-augmented-generation ollama

• Updated Feb 17, 2024
• Python

XML sitemap parser designed to extract and process millions of URLs while bypassing most modern anti-bot protections. Supports plain and compressed XML, unlimited nested sitemaps, multi-threading, multiple inputs, CloudScraper integration, fingerprint randomization, proxy/user agent rotation, auto stealth mode, and detailed monitoring.

sitemap robots-txt xml-parser xml-sitemap anti-bot sitemap-xml nested sitemap-files sitemaps bot-detection python-crawler website-scraper sitemap-url xml-sitemap-parser cloudflare-bypass proxy-rotation sitemap-parser sitemap-index cloudscraper sitemap-scraper

• Updated Apr 23, 2026
• Python

Plugin for website-scraper which returns html for dynamic websites using PhantomJS.

nodejs javascript scraper phantomjs hacktoberfest website-scraper

• Updated Dec 29, 2021
• JavaScript

Scraper for https://marvelsnapzone.com to retrieve metadata of Marvel SNAP cards.

game crawler scraper marvel website-scraper website-crawler marvel-characters crawler-python marvel-snap

• Updated Jul 1, 2024
• Python

Evaluate JavaScript on a URL through headless Chrome browser.

command-line cli-utilities evaluator web-browser eval web-crawling browser-automation headless-browsers data-scraping website-scraper web-data-extraction datascraping webscrapping web-scrapping data-scraper devtools-protocol pupeteer headless-browser scraping-tool playwright-alternative

• Updated Jun 18, 2021
• JavaScript

JSON collection of scraped file extensions, along with their description and type, from FileInfo.com

json scraper python3 scraped-data fileinfo file-extensions website-scraper

• Updated Jul 10, 2026
• Python