Apps with 'Web Crawler' feature Page 4 of 4

FandangoSEO is an SEO Cloud tool that consists of a Crawler, a Log Monitoring, and a Competitive Analysis Tool. It provides valuable reports to master SEO strategies, improve the crawl budget, and increase the mobile and desktop traffic.

Cloudtrellis scans your entire site for broken links, accessibility issues, and SEO improvements to ensure a flawless user experience.

Collect news articles easily from any sources in any language. API, MySQL, ElasticSearch integration.

Analyzes any public site for agent and LLM accessibility, scores agent-readability 0–100, explains each issue in plain English, and gives fixes for compliance.

LookaCrawler is a free open source local web crawler that strips over 73 percent of page bloat and extracts clean Markdown for LLMs. It runs fast HTTP and deep headless Chrome modes, caches results in SQLite, and ships a native MCP server over stdio and SSE for Claude Desktop...

Inout Scripts is a leading software development company that focuses primarily on clone scripts, Mobile Apps and e-business solutions. We have 21 scripts & 6 Mobile apps that are primarily used by small and medium ebusinesses in over 125...

Website audits for SEO, performance, structured data and AI visibility, with every fix ready to copy as a prompt.

Octocrawl turns public web pages into readable Markdown, tables and JSON for AI agents, RAG pipelines and research. Every page carries an Evidence Record: the final URL, HTTP status, robots.txt decision and hashes of what was read.

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

BlackWidow by SoftByte Labs Inc. scans websites (it's a site ripper). It can download an entire website, or download portions of a site.

Systematically monitor meeting minutes or meeting agendas and receive notifications when specific key terms or phrases appear.