Scrapy
Scrapy is a free and open-source web-crawling framework written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a g...
What is Scrapy?
Scrapy is a free and open-source web-crawling framework written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a general-purpose web crawler. It was developed and is maintained by Zyte formerly Scrapinghub, a web-scraping development and services company.
Scrapy Screenshots
Scrapy Features
Scrapy information
Supported Languages
- English
Comments and Reviews
Said about Scrapy as an alternative
All the infrastructure bits that come with it if you need. Its pretty much impossible to do a significant crawl all by yourself (proxy management hell), so being able to get things working first and then lean on the backend infrastructure the scrapy guys have is a major relief.
Requires lots of development to make it work. Has certain limitations when it comes to markup detection on a moderns sites.
Like ParseHub, Scrapy allows users to construct and run web scrapes. Unlike ParseHub, Scrapy is command-prompt based and has no GUI.
Tags
- Web Scraping
- framework
- web-scraping
Category
OS & UtilitiesList containing Scrapy
Core, Development & ServicesRecent user activities on Scrapy
DogmaDragon liked Scrapy
Danilo_Venom added Scrapy as alternative(s) to ScraperAPI- tomiolavitoivio added Web Scraper as a feature to Scrapyto
Excellent web crawler.