Attention

WARNING: From 9am on 19th August until 5pm on 2nd September there will be no access to the Stanage HPC cluster.

We will send an email to notify you when Stanage is back online and available for job submission.

Using the HPC Systems

If you have not used a High Performance Computing (HPC) cluster, the Linux operating system or even a command line before this is the place to start. This guide will get you set up using the University’s clusters (Stanage and Bessemer) fairly quickly.

• What is High Performance Computing?
  • Introduction
  • Who can use or should use a HPC cluster?
  • What is a HPC cluster?
  • When should I use HPC?
  • When should I not use HPC?
  • How do I get started?
• Getting an Account
  • For Staff
  • For Students
  • Conditions for HPC access / usage
• Connecting to Bessemer using myApps (web browser)
• Connecting to a cluster using SSH
  • Connecting with a Password or SSH keys
  • Establishing a SSH connection
  • Suggested SSH clients
  • What if I cannot use the VPN or I need a persistent long term connection
  • What Next?
• Tips for working with MFA
  • Reuse SSH connections
  • SSH clients
  • Putty
• Filestores
  • Choosing the correct filestore
  • Home directories
  • Fastdata areas
  • Shared (project) directories
  • Scratch directories
  • Community areas for software
  • How to check your quota usage
  • If you exceed your filesystem quota
  • Recovering files from snapshots
• Transferring files
  • Transfers with SCP/SFTP
  • How to download files directly to the cluster
• Job Submission and Control
  • Introduction
  • Key Concepts
  • Job Submission / Control on Bessemer
  • Job Submission / Control on Stanage
  • Cluster job resource limits
  • Advanced / Automated job submission and management
  • Reference information and further resources
• Advanced Job Submission and Control
  • Introduction
  • Advanced Job Submission Methods
  • Advanced workflow management tools
• Activating software using Environment Modules
  • Overview and rationale
  • Basic guide
  • Searching for Modules
  • Behind the scenes
  • Convenient ways to set up your environment for different projects
  • Managing your own module files
  • Compiling software dependent on modules
  • Module Command Reference
• Choosing appropriate compute resources
  • Introduction
  • Cluster choice
  • Time Allocation Limits
  • CPU Allocation Limits
  • Memory Allocation Limits
  • Filestore Limits / file store performance characteristics
  • Special limits and alternative queues
• Installing software to the clusters
  • General background prequisites
  • Installing software from binaries
  • Installing software by compiling from source
  • Making installed software available to execute
  • Why should I install from source?
  • What alternative methods exist?
• Graphical Sessions (With Flight) on Stanage
  • Usage instructions
• HPC Gateway Service
  • Service description
  • Access conditions
  • Overview of the connection process
  • Specific usage examples