Containers Guide

Containers are new and generally exciting development in HPC workloads. Containers rely on existing kernel features to allow greater user control over what applications see and can interact with at any given time. For HPC Workloads, these are usually restricted to the mount namespace. Slurm allows container developers to create SPANK Plugins that can be called at various points of job execution to support containers. Slurm is generally agnostic to containers and can be made to start most, if not all, types.

Links to several container varieties are provided below:

Please note this list is not exhaustive as new containers types are being created all the time.


Container Types

Charliecloud

Charliecloud is stand alone user namespace container out of LANL to provide HPC containers. Charliecloud does does not have/require any Slurm integration as it requires no special permissions to run as a user.

Docker

Docker currently has multiple design points that make it unfriendly to HPC systems. The issue that usually stops most sites from using Docker is the requirement of "only trusted users should be allowed to control your Docker daemon" [Docker Security] which is not acceptable to most HPC systems.

Sites with trusted users can add them to the docker Unix group and allow them control Docker directly from inside of jobs. There is currently no support for starting or stopping docker containers directly in Slurm.

UDOCKER

UDOCKER is Docker feature subset clone that is designed to allow execution of docker commands without increased user privileges.

Kubernetes Pods (k8s)

Kubernetes is a container orchestration system that uses PODs, which are generally a logical grouping of containers for singular purpose.

There is currently no support for Kubernetes Pods in Slurm.

Shifter

Shifter is a container project out of NERSC to provide HPC containers with full scheduler integration.

Singularity

Singularity is hybrid container system that supports:

  • Slurm integration via Plugin. A full description of the plugin was provided in the SLUG17 Singularity Presentation.
  • User namespace containers via sandbox mode that require no additional permissions.
  • Users directly calling singularity via setuid executable outside of Slurm.


Last modified November 2018