Read Me
The following is the README from our GitHub repository.
Sheffield High Performance Computing Documentation
This is the source for the documentation of The University of Sheffield’s High Performance Computing cluster(s).
It is written in the reStructuredText (rst) format and the Sphinx tool is used to convert this to a set of HTML pages.
For a guide on the rst file format see this document.
Rendered Documentation
This website is currently automatically built from this repository:
each push to the master branch causes a GitHub Actions workflow, which:
Builds the documentation using Sphinx
Adds a
robots.txtfile to the web root to guide indexing and reference the sitemapAdds a Google Search Console verification file for ownership validation
Generates a sitemap using the
sphinx-sitemapplugin to support better SEODeploys the resulting HTML to GitHub Pages
Test builds
For a review build, an administrator creates a remote test_* branch from
current master. Open a pull request from the content branch into it, check
the build result and merge for preview review. After approval, publish the
original content branch through a separate PR into master. For a coordinated
release, combine related PRs on a dedicated test branch, remove preview-only
changes such as a test banner, then open a PR from that branch into master.
The workflow publishes the preview branch at
https://docs.hpc.shef.ac.uk/en/<branch>/ alongside /en/latest/ while
the test branch exists. For example, test_newsletter appears at
/en/test_newsletter/. See the
test-build guide
for branch naming, updates, removal and public access.
Redirects
The published site retains the /en/latest/ URL structure from its former
Read the Docs hosting so existing links keep working. The
deployment workflow moves the built pages into
that directory and copies rtd-root-redirect.html to
the web root, redirecting https://docs.hpc.shef.ac.uk/ to /en/latest/.
Local Sphinx builds serve the pages directly from the build root.
The alternate domain, docs.hpc.sheffield.ac.uk, uses the separate
sheffield_hpc_redirect repository.
Its CNAME associates that domain with an organisation-controlled GitHub Pages
site, whose HTML redirects point to https://docs.hpc.shef.ac.uk/en/latest/.
This allows both custom domains to remain configured on our Pages sites. Keep
that configuration in place while DNS points to GitHub Pages; see GitHub’s
domain verification guidance
for protection against domain takeovers.
For renamed pages, html_additional_pages in conf.py maps each
old page path, without .html, to a redirect template in _templates.
For example, nvcc-stanage-redirect.html
redirects the old NVIDIA compiler page to the CUDA page. Add the old path to
this mapping and set the template’s HTML refresh target to the new relative URL.
This generates the redirect at the old URL without adding an obsolete page to
the table of contents through a placeholder RST file.
How to Contribute
To contribute to this documentation, you need to clone both the main repository and its submodule hpc-examples to your machine, use:
git clone --recurse-submodules git@github.com:rcgsheffield/sheffield_hpc.git
Once you have the git repository locally on your computer, install uv version 0.13.5 or newer. uv manages the local Python environment.
Please see our Documentation Reference which is a valuable resource for admins of our documentation.
Follow the style guide for language, command examples, software pages and validation conventions. For copyable email-contact examples and automatic compose-link hover tips, see the email contact guidance.
Once you have made your changes and pushed them to GitHub you will need to Open a Pull Request. All changes to the repository should be made through Pull Requests, including those made by the people with direct push access.
If you make changes to hpc-examples:
Open a PR to
hpc-examples/mainfirst.Do not update the submodule pointer in
sheffield_hpcuntil the PR is merged.Once merged, update the submodule pointer and create a separate PR in sheffield_hpc.
Building the documentation
From the repository root, install the locked dependencies and build:
uv sync --locked
uv run --locked sphinx-build -b html -W -d _build/doctrees . _build/html
uv uses the Python version in .python-version and can download it if needed.
The HTML is written to _build/html. On systems with Make, make html runs
that same build. Warnings fail the build, as they do in CI.
For a local preview that rebuilds when files change, run:
uv run --locked sphinx-autobuild -b html -W -d _build/doctrees . _build/html
Or use make serve (make serve PORT=8080 to choose another port). Stop an
existing preview before running a separate build in the same worktree.
Existing contributors can follow the Tox-to-uv transition guide, including an old-to-new command table and dependency-maintenance instructions.
Check external links
Do this with:
uv run --locked sphinx-build -b linkcheck -W -d _build/linkcheck-doctrees . _build/linkcheck
The workflow summary starts with broken links and timeouts from the main check,
including source locations and error reasons. The same Markdown report is
available as review.md in the linkcheck-results artifact, alongside
Sphinx’s raw results. Working links are omitted from the report.
The manual-review list and generated sidebar report follow it in the summary.
GNU, NonGNU and x265.org are excluded because they repeatedly rate-limit
automated requests. The private documentation repository and specified University
resources requiring authentication are also excluded: check them in a signed-in
browser. Public GitHub and Sheffield pages remain checked automatically.
Each scheduled or manual workflow run lists the excluded URLs and their
source locations in its summary and the manual-link-review artifact.
Check these URLs manually. To generate the same report locally, run:
python .github/scripts/report-manual-links.py
The report is written to _build/linkcheck/manual-review.md.
Generated sidebar URLs that do not also occur in handwritten documentation are
checked separately. Their failures do not fail the main link check. The
upstream-sidebar-review artifact and workflow summary list their results and
generated source locations so package/module metadata can be corrected upstream.
URLs shared with handwritten documentation remain in the main check. The
upstream report is retained for 30 days; ignored URLs require manual review.
To run the separate check locally after installing the locked dependencies:
uv run --locked sphinx-build -b linkcheck -D sidebar_linkcheck=1 \
-d _build/sidebar-doctrees . _build/sidebar-linkcheck
Testing the building of the documentation
The validity of the reStructuredText in this repo and the ability to convert that to HTML with Sphinx can be tested in three ways:
Locally by contributors when they run e.g.
make htmlBy a GitHub Actions Workflow each time a contributor creates or updates a Pull Request.
By a GitHub Actions) Workflow on each push to the
masterbranch.
Updating documentation dependencies
Use uv add <package> (uv add --dev <tool> for development tools),
uv remove <package>, or uv lock --upgrade-package <package>. To upgrade
an exactly pinned direct dependency, use uv add 'package==new-version'.
Run uv sync --locked and the HTML build, then review and commit the relevant
pyproject.toml and uv.lock changes together. Do not edit uv.lock by hand.
Dependency updates do not change the Python version in .python-version.
Important files / folders
conf.py- Sphinx configuration file.pyproject.toml- Direct dependencies, development tools and supported Python versions.uv.lock- Exact resolved dependency versions; commit this when dependencies change..python-version- Python version used for local builds and CI.Makefile- Short commands for building, previewing and checking links.global.rst- A globally included file (goes into all pages) which is excluded from direct building using exclude_patterns inconf.py.referenceinfo/imports- sub-folder tree of files to be included by not directly built. This is excluded from direct building using exclude_patterns inconf.py._static/css/custom.css- custom CSS overrides for the theme.themes/sheffieldhpc- Sheffield HPC custom theme components (Sphinx HTML templates, media files, CSS etc…). This functions as an overlay to the default Sphinx RTD theme..github/workflows- GitHub Actions workflows for pull requests, pushes tomasterand link checking.
Custom Google Search Engine
As the built in Sphinx search is naive / poor, a custom Google CSE has been added (currently implemented as per https://github.com/rcgsheffield/sheffield_hpc/pull/1971).
This is implemented with the following steps:
Create Google custom search on console: https://cse.google.com/cse/all
Copy HTML snippet for Google custom search
Paste it into _templates/searchbox.html.
Configure html_sidebars to use searchbox.html in your document.
Ensure your templates_path is set to correctly source the templates directory.
Customise the theming, search domain and other settings at https://cse.google.com/cse/all if not done already.
Test the search is configured and functioning as desired.
Google Analytics 4 (GA4)
We use Google Analytics 4 (GA4) to track usage statistics for our documentation.
This helps us understand which pages are most visited, how users navigate the docs, and where improvements can be made.
GA4 is enabled via the sphinxcontrib-googleanalytics plugin.
The Privacy Policy is linked in the footer of every documentation page.
The tracking ID is set in conf.py:
googleanalytics_id = 'G-HTK1JMDS73'
googleanalytics_enabled = True
If tracking needs to be disabled, set:
googleanalytics_enabled = False
Test tracking using Google Tag Assistant or GA4’s real-time reporting in Google Analytics.
Making or using imported files from the referenceinfo/imports area
This area is intended to be used to contain .rst files which we wish to use in more than one location which can be imported. The general method for making use of imported files is as follows:
Make a new file to be imported within a sensible subdirectory within this area e.g.
/referenceinfo/imports/software/mysoftware/import.rstImport your new file into your main page with:
.. include:: /referenceinfo/imports/software/mysoftware/import.rstBuild the documentation and ensure that hierarchical elements are correct e.g. titles within toctrees must be correct to fit in the parent document properly.
Add a comment within the import / parent document to explain why the import is necessary if it is not immediately obvious.
(Re)-generating PNG images from Mermaid.js diagram definitions
Some diagrams, such as images/hpcgateway-sequence-diag.png
have been generated with mermaid-cli
and Mermaid.js diagram definitions such as images/hpcgateway-sequence-diag.mmd.
How to install mermaid-cli and regenerate one of these diagrams:
yarn add @mermaid-js/mermaid-cli
./node_modules/.bin/mmdc -i images/hpcgateway-sequence-diag.mmd -o images/hpcgateway-sequence-diag.png
Software guidance dates
Maintain software guidance dates as described in the contributing guide.