Using the HPC Systems

If you have not used a High Performance Computing (HPC) cluster, the Linux operating system or even a command line before this is the place to start. This guide will get you set up using the University’s clusters (Stanage and Bessemer) fairly quickly.

• What is High Performance Computing?
  • Introduction
  • Who can use or should use a HPC cluster?
  • What is a HPC cluster?
  • When should I use HPC?
  • When should I not use HPC?
  • How do I get started?
• Getting an Account
  • For Staff
  • For Students
  • Conditions for HPC access / usage
• Connecting to a cluster using SSH
  • Connecting with a Password or SSH keys
  • Establishing a SSH connection
  • Suggested SSH clients
  • What if I cannot use the VPN or I need a persistent long term connection
  • What Next?
• Tips for working with MFA
  • Reuse SSH connections
  • SSH clients
  • Putty
• Filestores
  • Choosing the correct filestore
  • Home directories
  • Fastdata areas
  • Shared (project) directories
  • Scratch directories
  • Community areas for software
  • How to check your quota usage
  • If you exceed your filesystem quota
  • Recovering files from snapshots
• Transferring files
  • Transfers with SCP/SFTP
  • How to download files directly to the cluster
• Job Submission and Control
  • Introduction
  • Key Concepts
  • Job Submission / Control on Stanage & Bessemer
  • Cluster job resource limits
  • Advanced / Automated job submission and management
  • Reference information and further resources
• Parallel Computing
  • Embarrassingly Parallel
  • Shared Memory Parallelism
  • MPI for Multi-node and Parallel Jobs
  • GPU Computing
  • An Overview of Parallel Computing Methods
• Advanced Job Submission and Control
  • Introduction
  • Advanced Job Submission Methods
• Advanced Job Profiling and Analysis
  • Accessing a Running Single-Node Slurm Batch job
  • Accessing a Running Multi-Node Slurm Batch job
• Activating software using Environment Modules
  • Overview and rationale
  • Basic guide
  • Searching for Modules
  • Behind the scenes
  • Convenient ways to set up your environment for different projects
  • Managing your own module files
  • Compiling software dependent on modules
  • Module Command Reference
• Choosing appropriate compute resources
  • Introduction
  • Cluster choice
  • Time Allocation Limits
  • CPU Allocation Limits
  • Memory Allocation Limits
  • Filestore Limits / file store performance characteristics
  • Special limits and alternative queues
• Installing software to the clusters
  • General background prequisites
  • Installing software from binaries
  • Installing software by compiling from source
  • Making installed software available to execute
  • Why should I install from source?
  • What alternative methods exist?
• Graphical Sessions (With Flight) on Stanage
  • Usage instructions
• HPC Gateway Service
  • Service description
  • Access conditions
  • Overview of the connection process
  • Specific usage examples