Polars

Polars is a highly performant DataFrame library for manipulating structured data. The core is written in Rust, but the library is also available in Python. Its key features are:

• Fast: Polars is written from the ground up, designed close to the machine and without external dependencies.
• I/O: First class support for all common data storage layers: local, cloud storage & databases.
• Easy to use: Write your queries the way they were intended. Polars, internally, will determine the most efficient way to execute using its query optimizer.
• Out of Core: Polars supports out of core data transformation with its streaming API. Allowing you to process your results without requiring all your data to be in memory at the same time
• Parallel: Polars fully utilises the power of your machine by dividing the workload among the available CPU cores without any additional configuration.
• Vectorized Query Engine: Polars uses Apache Arrow, a columnar data format, to process your queries in a vectorized manner. It uses SIMD to optimize CPU usage.