HPC Example Scripts

The hpc-examples repository is a public collection of useful scripts and workflows for HPC systems.

Integrating hpc-examples into sheffield_hpc

The hpc-examples repository is included as a Git submodule within sheffield_hpc. Site maintainers can reference or include content from hpc-examples within the sheffield_hpc repository to enhance documentation and provide users with tested examples.

Steps to Use hpc-examples

  1. Clone or Update the Repository

Clone the hpc-examples repository if it has not already been added, or pull the latest changes: To ensure the hpc-examples submodule is included:

git clone --recurse-submodules git@github.com:rcgsheffield/sheffield_hpc.git

If you’ve already cloned without submodules:

git submodule update --init --recursive

  1. Reference Examples in Documentation

Use Sphinx’s literalinclude directive to embed scripts directly from the hpc-examples submodule. For example:

.. literalinclude:: /hpc-examples/path/to/example_script.sh
   :language: bash
   :lines: 3-10

See more examples.

  1. Add download commands for individual files

wget https://raw.githubusercontent.com/rcgsheffield/hpc-examples/refs/heads/main/$PATH_TO_FILE

Where $PATH_TO_FILE is the relative path within hpc-examples , for example examples/slurm-ansys-2023R2.sh

  1. Ensure Consistency

  • Review the examples in hpc-examples to ensure they align with the configurations and best practices of Sheffield’s HPC systems.

  • If customisation is required, document the changes explicitly in the sheffield_hpc repository without modifying the original hpc-examples content.

How to Make Changes to hpc-examples and Reflect Them in sheffield_hpc

If you make changes inside the hpc-examples submodule (e.g. editing or adding a script), follow these steps to ensure both repositories stay in sync:

First, commit and push the changes in hpc-examples:

cd hpc-examples
git add <file>
git commit -m "Add/update example script"
git push

Wait for the hpc-examples pull request to be reviewed, approved, and merged into main before updating the submodule pointer in sheffield_hpc:

cd ..
git add hpc-examples
git commit -m "Update hpc-examples submodule pointer"
git push

This ensures sheffield_hpc tracks a public, stable commit of hpc-examples.

Important

When updating the submodule pointer to hpc-examples:

  • Only reference commits that have already been merged into the main branch.

  • Never commit a pointer to a branch-only or unpublished commit — this will cause submodule fetch failures in GitHub Actions workflows and for collaborators cloning the repo.

  • Local changes to hpc-examples will build and render correctly within your local clone of sheffield_hpc, even before they are pushed or merged.

  • You may temporarily point to a local commit of hpc-examples for testing, but do not push that pointer to sheffield_hpc until the referenced commit is available in the upstream hpc-examples/main branch.

Repository Structure

Directory Layout - see hpc-examples

  • docs/: Documentation files.

  • examples/: Example SLURM scripts and application workflows (referenced from sheffield_hpc).

  • tests/: Example SLURM test scripts (referenced from sheffield_hpc).

  • tools/: Utilities or helper scripts for HPC users.

  • scripts/: General-purpose scripts for users.

  • configs/: Example configuration files for applications or tools.

  • tutorials/: Scripts required for tutorials within our docmentation.

Naming Conventions

  • Use descriptive, kebab-case names for files and directories.

  • Include version numbers in filenames for version-specific examples.