Elegant Scraper and Crawler Framework for Golang

Declarative web scraping

A Chrome DevTools Protocol driver for web automation and scraping.

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

Extract structured data from web sites. Web sites scraping.

Second-order subdomain takeover scanner

🕸️ Crawl in the web network

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

Open source SEO auditing tool.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Download a large list of files concurrently

Go process used to crawl websites

Scraply a simple dom scraper to fetch information from any html based website

A fast, modern and intelligent proxy rotator perfect for crawling and scraping public data.

Fast, highly configurable, cloud native dark web crawler.

🌱 goClone - clone websites in seconds

A Content Discovery and Development Platform. Empowering Cybersecurity, AI, Marketing, and Finance professionals and researchers to discover, analyze, and interact with the web in all its dimensions.

Raven is a powerful and customizable web crawler written in Go.

Advanced declarative web scraping