Slurm

Slurm, Simple Linux Utility for Resource Management, is an open-source workload manager designed for Linux clusters of all sizes. It provides three key functions. First it allocates exclusive and/or non-exclusive access to resources (computer nodes) to users for some duration of time so they can perform work. Second, it provides a framework for starting, executing, and monitoring work (typically a parallel job) on a set of allocated nodes. Finally, it arbitrates contention for resources by managing a queue of pending work.

While other workload managers do exist, Slurm is unique in several respects:

Scalability: It is designed to operate in a heterogeneous cluster with up to tens of millions of processors. Performance: It can accept 1,000 job submissions per second and fully execute 500 simple jobs per second (depending upon hardware and system configuration). Free and Open Source: Its source code is freely available under the GNU General Public License. Portability: Written in C with a GNU autoconf configuration engine. While initially written for Linux, Slurm has been ported to a diverse assortment of systems. Power Management: Job can specify their desired CPU frequency and power use by job is recorded. Idle resources can be powered down until needed. Fault Tolerant: It is highly tolerant of system failures, including failure of the node executing its control functions. Flexibility: A plugin mechanism exists to support various interconnects, authentication mechanisms, schedulers, etc. These plugins are documented and simple enough for the motivated end user to understand the source and add functionality. Resizable Jobs: Jobs can grow and shrink on demand. Job submissions can specify size and time limit ranges. Status Jobs: Status running jobs at the level of individual tasks to help identify load imbalances and other anomalies.