Apps with 'Web Crawler' feature Page 2 of 3

All apps in Apps with 'Web Crawler' feature category. Use the filters below to narrow down your search. 
Copy a direct link to this comment to your clipboard
  1. Greenflare icon
     3 likes

    Open-Source Python Based SEO Web Crawler Free SEO Analysis Tool Made by SEOs for Digital Marketing Experts

    Analyse and improve your onsite SEO Audit and identify any server issues Extract any on-page information you need.

    • Discontinued
    • FreeOpen Source
    • Windows
    • Mac
    Greenflare screenshot 1
    71 alternatives
  2. InterroBot icon
     1 like

    InterroBot combines a desktop web crawler with full-text and field searching to help you get answers to just what the heck is going on with your website.

    • SEO Tool
    • PaidProprietary
    • Mac
    • Windows
    • Linux
    • Android
    • Android Tablet
    • Flathub
    • Flatpak
    Projects view with information feed to right.
    22 alternatives
  3. SiteCrawler icon
     5 likes

    SiteCrawler is a website downloading application that lets you capture entire sites or selected portions, like image galleries. It features powerful settings that no other application offers.

    SiteCrawler screenshot 1
    29 alternatives
  4. Sublist3r icon
     3 likes

    Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting.

    • FreeOpen Source
    • Linux
    • Online
    • Self-Hosted
    Sublist3r screenshot 1
    7 alternatives
  5. SEO Checker icon
     1 like

    Vovsoft SEO Checker is an easy to use website crawler, able to scan both small and large websites efficiently, while allowing you to find broken links, analyze the results in real-time, gather on-page SEO data, distinguish internal and external links, classify dofollow and...

    • FreemiumProprietary
    • Windows
    SEO Checker screenshot 1
  6. OpenTheatre icon
     3 likes

    It communicates with its own API written entirely using our custom built command-line web crawler designed to scrape information from trusted files, updated every day. The public web resources used are available on our open assets database, where anyone can contribute their open...

    • Discontinued
    • FreeOpen Source
    • Windows
    OpenTheatre screenshot 1
    62 alternatives
  7. Netlas.io icon
     28 likes

    Enriched and structured information about every host on the Internet. Non-intrusive internet scanner, whois, domain names & SSL certs database.

    Netlas.io screenshot 1
    7 alternatives
  8. Blue Crab icon
     1 like

    Blue Crab is a versatile app that you use to download the contents of a website to your Mac, in whole or in part, and then search it.

    Blue Crab screenshot 1
    9 alternatives
  9. Web scraping API with a flexible usage-based pricing that offers real browsers, rotating mobile proxies, and JavaScript rendering.

    • Web Scraping Tool
    • PaidProprietary
    • Mac
    • Windows
    • Linux
    • Online
    • BSD
    • Google Chrome
    • Software as a Service (SaaS)
    • Firefox
    Landing Page
    24 alternatives
  10. dexi.io icon
     13 likes

    Dexi is the most comprehensive web data processing tool for professionals. Extract, enrich & connect any data source to any other data source. Also known to be most advanced web scraping tool. All available in your browser - try it for free.

    dexi.io screenshot 1
    73 alternatives
  11. The primary focus of this web crawler is to make it easier for enterprise search integrators and developers when dealing with all types of crawling requirements. We wanted for ourselves something very portable, extensible, modular, and working with most search engines.

    • FreeOpen Source
    • Mac
    • Windows
    • Linux
    • Java Mobile
    1 alternatives
  12. The Audisto Crawler is a website crawling tool with a strong focus on SEO. It automatically detects and analyses all URLs from a website and points out optimization potential.

    • SEO Tool
    • PaidProprietary
    • Online
    • Software as a Service (SaaS)
    Audisto Crawler screenshot 1
    39 alternatives
  13. The best choice for network management with powerful and professional network tools focused in performance, accuracy and simplicity.

    • FreemiumProprietary
    • Android
    • Chrome OS
    • Android Tablet
    NetMan: Network Tools & Utils screenshot 1
    NetMan: Network Tools & Utils screenshot 2
    NetMan: Network Tools & Utils screenshot 3