AlternativeTo Logo

Apps with 'Crawler' feature

  1. Pulno is a website audit tool which checks for SEO related issues and sends comprehensible tips to improve on-page SEO with page speed optimizer (CSS and image file optimization), meta tags and unique text analyzer.

  2. import.io is a web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required.



  3. Netpeak Spider is an SEO crawler for a day-to-day SEO audit, fast issue check, comprehensive analysis, and website scraping.

  4. A program to download photo and video from Reddit, Twitter, Instagram, and any site (using plugins).

  5. A web-based crawler with real-time crawl feedback.

    Advanced, fast & flexible SEO website crawler that can help identify technical or architectural issues with any site.

  6. OpenRPA icon
    • FreeOpen Source
    • Windows
    • Online
    • Microsoft Edge
    • Google Chrome
    • Python
    • AutoHotkey
    • JavaScript
    • Software as a Service (SaaS)
    • Firefox

    OpenRPA automates repetitive tasks performed by humans to cut costs, eliminate keying errors, speed up processes, and easily link applications together. These tasks can be converted into a business process and then automated.

  7. Octoparse is a modern visual web data extraction software. Both experienced and inexperienced users would find it easy to use Octoparse to bulk extract information from websites, for most of scraping tasks no coding needed.

  8. Common Crawl builds and maintains an open repository of web crawl data that can be accessed and analyzed by anyone

  9. Extracty can extract any web data and create an API to the webpage's information.

    Discontinued

    The company do not host an online backend anymore, so it's not possible to execute any scripts online. The website has a functional front end.

    Screenshot
  10. Web scraping including cloud.

    No screenshots yet
  11. ProxyCrawl helps you stay anonymous while crawling the web, web crawling protection the way it should be.

  12. JetOctopus is a full-service SEO website crawler and log analyzer that helps you get valuable on-page data on every URL and improve SEO performance. Turn your attention to the detailed reports and find website issues other tools miss.

    Crawl report.
Crawler detects crucial for Google technical problems, and provides you with a scale of each problem. So an average SEO manager knows where to start fixing.
  13. A1 Website Analyzer can crawl websites and collect data to be used for finding site and SEO problems. Filter results to see which URLs are blocked by robots.txt, which are noindex, nofollow, canonical to self, canonical to other, response codes etc.

  14. SEO Tester Online is the web marketing suite that helps marketers, e-commerce and web agencies in 25 countries around the world to monitor and improve the organic visibility of its website, thanks to innovative and guided tools.

    No screenshots yet
  15. Sitechecker icon
    • FreemiumProprietary
    • Online
    • Self-Hosted
    • Google Chrome
    • Software as a Service (SaaS)

    Sitechecker is a website SEO audit tool with detailed SEO reports. Sitechecker enables users to conduct a comprehensive website audit and record technical mistakes for the whole domain and specific URLs.