StormCrawler is described as 'Open source SDK for building distributed web crawlers with Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java' and is an app. There are more than 10 alternatives to StormCrawler for a variety of platforms, including Windows, Linux, Mac, Web-based and BSD apps. The best StormCrawler alternative is Scrapy, which is both free and Open Source. Other great apps like StormCrawler are Lookyloo, Flyscrape, Crawlbase and Scraperr.
Scrapy is a free and open-source web-crawling framework written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a general-purpose web crawler. It was developed and is maintained by Zyte formerly Scrapinghub, a web-scraping...
Scrapy is the most popular Windows, Mac & Linux alternative to StormCrawler.
Scrapy is the most popular Open Source & free alternative to StormCrawler.
Lookyloo is a web interface that allows users to capture a website page and then display a tree of domains that call each other.
Lookyloo is the most popular Web-based alternative to StormCrawler.
Flyscrape is a standalone and scriptable web scraper, combining the speed of Go with the flexibility of JavaScript. — Focus on data extraction rather than request juggling.
Crawlbase, formerly ProxyCrawl, helps you stay anonymous while crawling the web, web crawling protection the way it should be.
Scraperr is a self-hosted web application that allows users to scrape data from web pages by specifying elements via XPath. Users can submit URLs and the corresponding elements to be scraped, and the results will be displayed in a table.
Scraperr is the most popular Self-Hosted alternative to StormCrawler.
Open-source, extensible web crawler designed for large-scale, archival-quality web archiving, preserves digital artifacts, supports modular plugins, distributed crawling, detailed monitoring, scheduling, and exports data in standardized formats for preservation.
Mixnode is a fast, flexible, massively scalable platform to extract and analyze data from the web.
Almost everyone thinks Mixnode is a great StormCrawler alternative.
Mixnode is the most popular commercial alternative to StormCrawler.
Apache Nutch is a highly extensible and scalable open source web crawler software project.
Kaddara is a platform designed for professionals who need fresh leads to run their business and whose business is affected by how competitors operate.
Kaddara is the most popular SaaS alternative to StormCrawler.
Switzerland
Germany
EU
United States
