On the Hellgate Research cluster we use Apptainer, an OCI-compliant containerization platform. This allows it to work well with Docker and other container images. Full user documentation can be found at:
Containers:
Containers offer a flexible and secure way to run applications. This enables the creation of environments for various research programs that may not be able to run on Hellgate's native environment. On Hellgate there is a shared directory with already built containers that all users have access to.
- /mnt/beegfs/projects/resources/Containers
Apptainer has two parts to it, a definition (def) file and a container binary (sif).
Definition File:
A definition file is a set of blueprints explaining how to build the container. It will include specifics about the base OS to build with or a container to start from, software to be installed, environment variables, files to add from the host system and container metadata. These files usually end with the .def extension to signify it is a definition file.
Definition files are typically divided into two parts: Header and Section(s)
- Header: defines core OS to build
- Sections: optional - executed at build time to further configure container
If you have an already existing container and want to find its definition file you can use:
- apptainer inspect --deffile <container_name.sif>
Building an Apptainer Container:
Building an Apptainer container involves using the apptainer build command, a versatile tool for
creating containers. You can use it to download and assemble containers from external OCIcompliant resources (like Docker Hub) or creating a container from scratch with a definition file.
- apptainer build <container_name.sif> <def_file.def>
Apptainer Images are usually given a .sif extension to signify it as a container.
Once a container is built user will lose write permissions, to ensure the container is working correctly users can start by building a sandbox, making necessary changes and running tests, and converting to a container once done.
Creating an Apptainer Sandbox:
Sandboxing provides users with a read-write container that is useful when developing and testing. To build one users will run this command from the directory they wish to have the container in.
- apptainer build --sandbox <sandbox_name> <def_file.def>
The apptainer shell --writable command is used to start an interactive shell within a sandbox. This allows users to easily modify the container's filesystem, install new software and make other necessary changes.
- apptainer shell --writable <sandbox_name>
If that command throws an error about having insufficient permissions can use the -fakeroot command to get escalated privileges.
- apptainer shell —fakeroot <sandbox_name>
If users need to remove a sandbox they will first have to change the permissions of the files before removing.
- chmod -R u_rwX <sandbox_name>
- rm -R <sandbox_name>
Converting Into a Container:
When converting the finished sandbox into a container it will create a compressed, read only Singularity Image File (SIF) in the same directory.
- apptainer build <container_name.sif> <sandbox_name>
It is recommended to create a .def file to document what was configured on the container, allowing for easier reproducibility in the future.
Using Apptainer: Shell
When using Apptainer one of the ways to use it is the shell command. This allows users to start an interactive shell within a container. It is particularly useful for exploring the environment and debugging. Running this command will place the user inside the container's environment and command can be executed as if you were inside a Linux environment.
- apptainer shell <container_name.sif>
To exit the shell users can type 'exit' or press Ctrl+D.
Using Apptainer: Exec
Another way to use Apptainer is the exec command that allows user to execute a specific command within a container. This is helpful when running a single command or script inside the container environment without launching the interactive shell.
- apptainer exec <container_name.sif> <command>
Using Apptainer: Run
The third option when using Apptainer is the run command, used to launch an Apptainer container and execute a runscript if one is defined for that container.
- apptainer run <container_name.sif>
Using Apptainer: Binding Directories
The final option when using Apptainer is the --bind option. This is used to bind host system directories to directories within the containers. Allowing users to access files from their host system inside the container for processing.
- apptainer shell --bind <host_path>:<container_path> <container_name.sif>