Scraperr

Scraperr is a self-hosted web application that allows users to scrape data from web pages by specifying elements via XPath. Users can submit URLs and the corresponding elements to be scraped, and the results will be displayed in a table.

From the table, users can download an Excel sheet of the job's results, along with an option to rerun the job.

Features:

Submitting URLs for Scraping

• Submit/queue URLs for web scraping
• Add and manage elements to scrape using XPath
• Scrape all pages within same domain
• Add custom JSON headers to send in requests to URLs
• Display results of scraped data

Managing Previous Jobs

• Download CSV containing results
• Rerun jobs
• View status of queued jobs
• Favorite and view favorite jobs

User Management

• User login/signup to organize jobs (optional)

Log Viewing

• View app logs inside of web UI

Statistics View

• View a small statistics view of jobs ran

AI Integration

• Include the results of a selected job into the context of a conversation
• Currently supports: Ollama, OpenAI