Selecting GPU's in Fox

Fox can support different node configurations which means that as Fox develops new nodes can be added, which are different from the initial nodes that Fox was installed with. When it comes to CPUs and memory this distinction will usually result in quicker runtime for existing projects, but for GPUs the story is a bit different. By allowing research groups to contribute to the procurement of new Fox nodes we also open up the possibility that some groups have different computational demands and budgets.

Selecting GPU's in Slurm

Luckily Slurm already supports specifying which GPU type to use in the many different GPU flags. To select a specific GPU use the Slurm GPU's flag of choice and add the type of the GPU before the number of GPUs, as follows:

$ srun <other flags...> --gpus=a100:1

In the above command we ask for 1 GPU with its type set to a100. We could ask for multiple GPUs by increasing the number, --gpus-per-task=a100:4, and we could combine GPUs to ask for GPUs of different types, --gpus=a100:1,rtx30:1.

If no GPU type is specified, e.g. when using --gpus=1, any available GPU can be assigned to the job.

Available GPUs

In the table below one can see the different GPUs, their name to use with the Slurm GPU flags, the partition and the number of GPUs available of the different types.

GPU type Slurm name Partition #Available nodes
A100 40GB PCIe a100 accel 2
A100 40GB PCIe a100 hudel & accel 1
RTX3090 24GB PCIe rtx30 ifi_accel & accel 3

Note that some GPUs have multiple partitions associated, this allows the GPUs to be selected from all listed partitions. E.g. to select an RTX3090 we could use --partition=accel --gpus=rtx30:1 or we could use --partition=ifi_accel --gpus=rtx30:1. This also means that for a specific partition the default GPU selected with --gpus=1 will change according to what is available in the partition.