Nodes Overview¶
Apocrita runs a variety of different job types, and the cluster is comprised of different node types and queues to accommodate this. Unless you have a specific technical reason, you should avoid selecting a specific node type to run your job on. Some nodes are restricted depending on the source of funding.
Hyperthreading is not enabled across the cluster. The "Threads" column within the below tables has been added for completeness.
The following nodes are open access, allowing jobs from any user.
| Open Access | Required variables | Count | Cores | Threads | RAM | Accessible RAM | Arch |
|---|---|---|---|---|---|---|---|
| ddg (volta) | gpu | 2 | 48 | 48 | 384GB | ~362GB | Intel |
| ddy | 56 | 48 | 48 | 384GB | ~362GB | Intel | |
| sbg (ampere) | gpu | 1 | 48 | 48 | 384GB | ~362GB | Intel |
| sbg (volta) | gpu | 2 | 32 | 32 | 384GB | ~362GB | Intel |
| sdx | 52 | 36 | 36 | 384GB | ~362GB | Intel | |
| smf | highmem | 8 | 24 | 24 | 768GB | ~741GB | Intel |
| srm | highmem | 2 | 36 | 36 | 768GB | ~741GB | Intel |
| vhm | highmem | 4 | 64 | 64 | 3TB | ~2.95TB | Intel |
| Total | 127 | 5288 | 5288 | 63.4TB |
The following nodes are restricted to specific groups controlled by the purchasers of the nodes. Some are bulk purchases on behalf of a faculty (e.g Medicine & Dentistry and Science & Engineering), and jobs requested by members of the faculty will automatically be eligible for these nodes. Other nodes have been purchased by smaller research groups and requests for access to these nodes will need to be confirmed by the owner of the node (typically the P.I.) before access will be granted.
| Restricted Access | Required variables | Count | Cores | Threads | RAM | Accessible RAM | Arch |
|---|---|---|---|---|---|---|---|
| ddy | 127 | 48 | 48 | 384GB | ~362GB | Intel | |
| rdg (ampere) | gpu | 12 | 48 | 48 | 1024GB | ~988GB | Intel |
| rdg (hopper) | gpu | 1 | 32 | 32 | 512GB | ~491GB | Intel |
| sbg (ampere) | gpu | 2 | 32 | 32 | 384GB | ~362GB | Intel |
| sbg (ampere) | gpu | 1 | 48 | 48 | 384GB | ~362GB | Intel |
| sbg (ampere) | gpu, cluster=andrena | 16 | 48 | 48 | 384GB | ~362GB | Intel |
| sdv serial | 4 | 24 | 24 | 192GB | ~177GB | Intel | |
| sdv parallel | infiniband=sdv-ii | 8 | 24 | 24 | 192GB | ~177GB | Intel |
| srm | highmem | 1 | 48 | 48 | 1536GB | ~1483GB | Intel |
| xdg (hopper) | gpu | 2 | 64 | 64 | 1024GB | ~988GB | Intel |
| Total | 174 | 8048 | 8048 | 74.8TB |
Owned nodes complex
Users with access to owned nodes can ensure jobs run on an owned node by
requesting -l owned in the job script or on the command line e.g.
qlogin -l owned
The following list shows the CPU instruction sets that are supported by Apocrita nodes. To see whether a particular CPU instruction set is supported, you can check the CPU flags on a specific node.
- SSE
- SSE2
- SSSE3
- SSE4
- AVX
- AVX2
- AVX-512