A next-generation crawling and spidering framework.

cli crawler headless hacktoberfest web-spider spider-framework gocrawler

• Updated Nov 9, 2026
• Go

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

crawler spider web-crawler jsoup xpath webcrawler webspider web-spider spider-flow

• Updated Jun 14, 2023
• Java

简单易用的Python爬虫框架，QQ交流群：597510560

python crawler multi-threading spider multiprocessing web-crawler proxies python-spider web-spider

• Updated Jun 10, 2022
• Python

SpiderSuite (web security crawler) releases, wiki and roadmap

crawler gui spider cplusplus cybersecurity recon bugbounty pentest webcrawler information-gathering security-tools web-spider osint-tool cybersecurity-tools

• Updated Aug 13, 2026

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

ruby crawler scraper web spider web-crawler web-scraper web-scraping web-spider spider-links

• Updated Jan 12, 2026
• Ruby

微信文章爬虫，用于采集、整理和归档微信公众号公开文章元数据，欢迎一起学习讨论。A Windows research workbench for collecting, organizing, and archiving public WeChat article metadata.

weixin wechat research-tool content-analysis web-spider academic-research wechat-articles

• Updated Sep 28, 2026
• Python

天眼查爬虫&企查查爬虫，指定关键字爬取公司信息

proxy python3 ua qichacha web-spider tianyancha

• Updated Feb 16, 2023
• Python

ACHE is a web crawler for domain-specific search.

web-crawler web-scraping hacktoberfest web-spider focused-crawler domain-specific-search web-search

• Updated Aug 31, 2025
• Java

The simple, easy to use command line web crawler.

python crawler web-crawler crawling python3 web-spider

• Updated Aug 8, 2024
• Python

The unix-way web crawler

go cli golang crawler web-crawler web-scraping pentesting pentest golang-application web-spider pentest-tool unix-way

• Updated Sep 11, 2026
• Go

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

lightweight elasticsearch crawler spider web-crawler scraping crawling web-scraping web-spider

• Updated May 19, 2021
• Go

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

golang crawler framework web-crawler scraping crawling web-spider

• Updated May 31, 2020
• Go

旨在将网易云、酷狗、QQ、酷我等各音乐平台集于一体

python music-player web-crawler web-spider music-downloader music-download-script qq-music wangyiyunmusic kugou-music kuwo-music music-robot

• Updated Nov 26, 2022
• Python

Ignareo the Carillon, a web crawler/spider template of ultimate high concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns! (ISML=international saimoe; 2022 ISML is last ISML)

python http microservice high-performance web-crawler concurrency distributed asyncio gevent web-spider isml sukasuka chtholly sukamoka ignareo tiat

• Updated Nov 5, 2026
• Python

A simple distributed crawler for zhihu && data analysis

python crawler spider web-crawler python-crawler web-spider

• Updated Dec 7, 2022
• Python

Web spider as a service, spider on serverless

javascript api aws-lambda serverless web-spider

• Updated Dec 7, 2022
• JavaScript

Tool made to automate tasks of pentesting.

security ipv6 ipv4 password-strength password-safety ssl-certificate infosec information-security ssl-certificates mac-address hacking-tool base64-encoding base64-decoding network-security security-tools ssl-cert social-engineering web-spider hacking-framework hexadecimal-converter

• Updated May 20, 2026
• Python

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

anti-bot-page anti-bot python-web-crawler python-crawler python-spider web-crawler-python web-spider python-scraper tls-client python-web-scraper cloudflare-bypass scraping-python python-web-scraping crawling-python web-scraping-python anti-bot-detection cf-clearance cloudflare-scraper web-scraping-api python-tls-client

• Updated Feb 23, 2026
• Python

talospider - A simple,lightweight scraping micro-framework

python crawler spider crawling web-spider

• Updated Oct 30, 2026
• Python

网络爬虫模拟登陆bilibili 滑动验证码的破解 弹幕发送 2018-10-9

selenium python3 bilibili web-spider analog-login

• Updated Mar 22, 2019
• Python