• Simplescraper icon
  • Wintr icon
  • UI.Vision RPA icon
  • TagUI icon
  • ScrapeMate icon
  • +28

Data mining

Francisco Ferioli MarcoFrancisco Ferioli Marco33 appsUpdated
Copy a direct link to this comment to your clipboard
  1. A web scraper that's fast, intuitive and 100% free to use in the browser. Download data from websites and tables in seconds.

    Cost / License

    • Freemium
    • Proprietary

    Application type

    Platforms

    • Online
    • Google Chrome
    • Software as a Service (SaaS)
    Simplescraper screenshot 1
    Simplescraper screenshot 1
    Simplescraper screenshot 2
    +1
    Simplescraper screenshot 3
  2. Wintr icon
     Like

    Free proxy service and web scraping API that allows you to scrape and parse any webpage's HTML with Cheerio to turn it into a personalized item dataset.

    Cost / License

    • Free
    • Proprietary

    Application type

    Platforms

    • Online
    Wintr screenshot 1

Linux based web scrapping

  1. Open-source task and test automation tool and Selenium IDE. The UI.Vision RPA software is a browser extension that can do desktop automation as well! Use it for web automation, form filling, screen scraping and Robotic Process Automation (RPA).

    Cost / License

    • Freemium
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • Google Chrome
    • Mozilla Firefox
    UI.Vision RPA software is a browser extension, but it can do desktop automation as well!
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  2. TagUI icon
     Like

    The goal of UI (user interface) automation is to reproduce cognitive interactions that you have with websites or your desktop, so that your computer can do it for you, base on your schedule or conditions.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • Google Chrome
    TagUI screenshot 1
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  3. Scraping assistant tool. Editing and maintaining CSS/XPath selectors across webpages. Available as a Chrome/Chromium and a Firefox extensions.

    Cost / License

    • Free
    • Open Source

    Platforms

    • Mac
    • Windows
    • Linux
    • Google Chrome
    • Chromium
    • Firefox
    ScrapeMate screenshot 1
    ScrapeMate screenshot 1
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  4. Pincers icon
     Like

    Web Automation API that runs on top of WebDriver.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  5. A lightweight Python library that uses Webkit to enable easy scraping of dynamic, Javascript-heavy web pages

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Linux
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  6. ACHE is a web crawler for domain-specific search.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
  7. Textricator is a tool for extracting text from computer-generated PDFs and generating structured data . If you have a bunch of PDFs with the same format (or one big, consistently formatted PDF) and you want to extract the data to CSV or JSON,.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    Textricator screenshot 1
    Textricator screenshot 1
    Textricator screenshot 2
    +2
    Textricator screenshot 3
  8. OpenSource Web Media Server to browse and stream any video file format supported by ffmpeg with easy web interface for play on any platform with html5 browser, dnla or kodi plugin.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Linux
    • Self-Hosted
    PHPMediaServer screenshot 1
    PHPMediaServer screenshot 1
    PHPMediaServer screenshot 2
    +2
    PHPMediaServer screenshot 3

Web based web scrapping

  1. Portia icon
     Like

    Portia is an open source visual scraping tool, allows you to scrape websites without any programming knowledge required! Simply annotate pages you're interested in, and Portia will create a spider to extract data from similar pages.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Linux
    • Vagrant
    • Docker
    Portia screenshot 1
    Portia screenshot 1
    Portia screenshot 2
    +2
    Portia screenshot 3
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  2. Apify icon
     Like

    Cloud-based platform for extracting data and automating website workflows, featuring headless browser support, advanced web crawling, reusable code acts and scalable storage.

    Cost / License

    • Freemium
    • Open Source

    Application type

    Platforms

    • Online
    Web scraper results
    Running an actor in console
    Task configuration
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  3. artoo.js icon
     Like

    artoo.js is a piece of JavaScript code meant to be run in your browser's console to provide you with some scraping utilities.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Online
    • Self-Hosted
    • Google Chrome
    • JavaScript
    • Node.JS
    artoo.js screenshot 1
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

Linux based data mining

  1. R is a free software environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories (formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be consider.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • BSD
    R (programming language) screenshot 1
    On Unix
    On Mac
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  2. Scrapy icon
     Like

    Scrapy is a free and open-source web-crawling framework written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a general-purpose web crawler. It was developed and is maintained by Zyte formerly Scrapinghub, a web-scraping...

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • BSD
    Scrapy screenshot 1
    Scrapy screenshot 1
    Francisco Ferioli MarcoFrancisco Ferioli Marco’s note

    Dataset scraping: yes.

  3. Orange icon
     Like

    Orange is an open-source, cross-platform data mining and machine learning suite. It features visual programming as intuitive means of combining data analysis and interactive visualization methods into powerful workflows.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    A data mining workflow.
    Hierarchical clustering.
    Image analytics.
    +6
    Analysis of misclassified data instances.
  4. KNIME icon
     Like

    Knime is a java open-source, cross-platform application which name means "Konstanz Information Miner". It is actually used extensively for data mining, data analysis and optimization. It can be downloaded as the core application itself(Knime Desktop), or the whole SDK...

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    Knime on MAC OSX
    Data analysis and visualisation
    managing extension and addons
    +1
    Visualization,
  5. Beaker icon
     Like

    The Beaker Notebook is a new open source tool for research and data science. It's advanced UI allows you to focus on your data and your science, instead of getting frustrated by your tool. We designed it to be polyglot from the ground up.

    Cost / License

    • Free
    • Open Source

    Platforms

    • Mac
    • Windows
    • Linux
    Beaker screenshot 1
    Beaker screenshot 1
    Beaker screenshot 2
    +1
    Beaker screenshot 3
  6. SpagoBI icon
     Like

    SpagoBI is the only entirely Open Source Business Intelligence suite. It covers all the analytical areas of Business Intelligence projects, with innovative themes and engines. SpagoBI offers a wide range of analytical tools: Reporting, Multidimensional Analysis (OLAP), Charts...

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Windows
    • Linux
    Charts/Dashs
    Data Mining
    GIS
  7. WEKA icon
     Like

    Weka is a collection of machine learning algorithms for data mining tasks; with its own GUI.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    WEKA screenshot 1
    WEKA screenshot 1
    WEKA screenshot 2
  8. Stan icon
     Like

    Stan is a probabilistic programming language for data analysis, enabling automatic inference for a large class of statistical models.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • Python
    • R (programming language)
    • Julia
    • MATLAB
  9. dradis icon
     Like

    Dradis is an open source framework to enable effective information sharing, specially during security assessments.

    Cost / License

    • Freemium
    • Open Source

    Platforms

    • Mac
    • Windows
    • Linux
    • BSD
    dradis screenshot 1
    dradis screenshot 1
  10. sn0int icon
     Like

    sn0int is a semi-automatic OSINT framework and package manager. It was built for IT security professionals and bug hunters to gather intelligence about a given target or about yourself. sn0int is enumerating attack surface by semi-automatically processing public information and...

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • BSD
    sn0int screenshot 1
  11.  Like

    The caret package (short for _C_lassification _A_nd _RE_gression _T_raining) is a set of functions that attempt to streamline the process for creating predictive models. data splitting pre-processing feature selection model tuning using resampling.

    Cost / License

    • Free Personal
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • R (programming language)
  12. Pyspread icon
     Like

    Pyspread is a non-traditional spreadsheet application that is based on and written in the programming language Python.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    Pyspread screenshot 1
  13. DataMelt icon
     Like

    DataMelt (or DMelt) is a program for numeric computation, statistics, data analysis and data visualization.

    Cost / License

    • Free Personal
    • Open Source

    Platforms

    • Mac
    • Windows
    • Linux
    • Android
    Kinematics of top decays
    A candlestick chart used to describe price movements of a security, derivative, or currency.
    Charts based on the HChart
  14. ELKI icon
     Like

    ELKI: "Environment for Developing KDD-Applications Supported by Index-Structures" is a development framework for data mining algorithms written in Java. It includes a large variety of popular data mining algorithms, distance functions and index structures.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    ELKI screenshot 1
    ELKI screenshot 1
    ELKI screenshot 2
  15. Widestage icon
     Like

    Light weight open source reports tool, that allows users to create their own HTML reports and dashboards just dragging and dropping elements, powered by a semantic layer.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Mac
    • Windows
    • Linux
    • Self-Hosted
    Data explorer, allows you to drag and drop elements and create queries on the fly.
    Choose between grid, charts, gauges, etc... to visualise your data
    Mix up in a HTML page different reports and make them work together applying common filters

Web based data mining

  1. InfraNodus can visualize any research notes, ideas, texts, even the Google search results on a certain topic as a text network. The words are the nodes and their co-occurrences are the connections between them.

    Cost / License

    • Paid
    • Proprietary

    Application type

    Platforms

    • Online
    • Software as a Service (SaaS)
    InfraNodus screenshot 1
    InfraNodus screenshot 1
    InfraNodus screenshot 2
  2. Apache Mahout is an Apache project to produce free implementations of distributed or otherwise scalable machine learning algorithms on the Hadoop platform. Mahout is a work in progress; the number of implemented algorithms has grown quickly, but there are still various...

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Linux
    • Online
  3. ggraptR icon
     Like

    ggraptR is an open source R package providing a GUI for visualization. It is based on principles of visualization analysis by Tamara Munzner, and also acts as a wrapper for functionality implemented in the grammar of graphics for R, ggplot2.

    Cost / License

    • Free
    • Open Source

    Application type

    Platforms

    • Windows
    • Linux
    • Online
    • Self-Hosted
    ggraptR screenshot 1
    ggraptR screenshot 1
  4. Facilitates web scraping and lead generation using no-code bots across LinkedIn, Twitter, Facebook, and more. Cloud-based APIs enhance productivity.

    Cost / License

    • Paid
    • Open Source

    Application type

    Platforms

    • Online
    • Mozilla Firefox
    PhantomBuster screenshot 1
    PhantomBuster screenshot 1
    PhantomBuster screenshot 2
    +1
    PhantomBuster screenshot 3
  5. REPODS icon
     Like

    REPODS is an online data warehouse service for managing & analyzing data histories in data pods. Data can be imported via various interfaces. IoT devices can also stream data directly to a data pod for cross-analysis with other data warehouse data.

    Cost / License

    • Freemium
    • Open Source

    Platforms

    • Mac
    • Windows
    • Linux
    • Online
    • Chrome OS
    • Tableau
    • QlikView

No comments so far, maybe you want to be first?