Attention

Following the recent operating system upgrade, software listings and module load commands now reflect the upgraded system. Some examples and notices are still being updated and may contain outdated guidance.

Using the HPC Systems

If you have not used a High Performance Computing (HPC) cluster, the Linux operating system or even a command line before this is the place to start. This guide will get you set up using the University’s cluster(s) fairly quickly.

• What is High Performance Computing?
  • Introduction
  • Who can use or should use a HPC cluster?
  • What is a HPC cluster?
  • When should I use HPC?
  • When should I not use HPC?
  • How do I get started?
• Getting an Account
  • For Staff
  • For Students
  • Conditions for HPC access / usage
• Connecting to a cluster using SSH
  • Connecting with a Password or SSH keys
  • Establishing a SSH connection
  • Suggested SSH clients
  • What if I cannot use the VPN or I need a persistent long term connection
  • What Next?
• Tips for working with MFA
  • Reuse SSH connections
  • SSH clients
  • Putty
• Filestores
  • Choosing the correct filestore
  • Home directories
  • Fastdata areas
  • Shared (project) directories
  • Scratch directories
  • Community Software Areas
  • How to check your quota usage
  • If you exceed your filesystem quota
  • Recovering files from snapshots
• Transferring files
  • Transfers with SCP/SFTP
  • How to download files directly to the cluster
• Job Submission and Control
  • Introduction
  • Key Concepts
  • Job Submission / Control using SLURM
  • Cluster job resource limits
  • Advanced / Automated job submission and management
  • Reference information and further resources
• Parallel Computing
  • Embarrassingly Parallel
  • Shared Memory Parallelism
  • MPI for Multi-node and Parallel Jobs
  • GPU Computing
  • An Overview of Parallel Computing Methods
• Priority access to HPC resources
  • HPC High-Priority Service
  • Getting a High-Priority Service account
  • Using the High-Priority Service
  • Troubleshooting High-Priority access
• Advanced Job Submission and Control
  • Introduction
  • Advanced Job Submission Methods
• Advanced Job Profiling and Analysis
  • Accessing a Running Single-Node Slurm Batch job
  • Accessing a Running Multi-Node Slurm Batch job
• Activating software using Environment Modules
  • Overview and rationale
  • Basic guide
  • Searching for Modules
  • Behind the scenes
  • Convenient ways to set up your environment for different projects
  • Managing your own module files
  • Compiling software dependent on modules
  • Module Command Reference
• Choosing appropriate compute resources
  • Introduction
  • Cluster choice
  • Time Allocation Limits
  • CPU Allocation Limits
  • Memory Allocation Limits
  • Filestore Limits / file store performance characteristics
  • Special limits and alternative queues
• Installing software to the clusters
  • General background prequisites
  • Installing software from binaries
  • Installing software by compiling from source
  • Making installed software available to execute
  • Why should I install from source?
  • What alternative methods exist?
• Graphical Sessions on Stanage
• HPC Gateway Service
  • Service description
  • Access conditions
  • Overview of the connection process
  • Specific usage examples