Attention

Following the recent operating system upgrade, software listings and module load commands now reflect the upgraded system. Some examples and notices are still being updated and may contain outdated guidance.

Python

Use module-provided Python with venv for ordinary Python projects and when working with packages supplied by HPC modules. A virtual environment lets you install additional Python packages without changing the shared installation.

For environments managed by Conda, including packages with non-Python dependencies, see Conda on Stanage.

Documentation: https://www.python.org/doc/

Loading Python

After connecting to Stanage (see Establishing a SSH connection), start an interactive session with the following command:

srun --pty bash -i

Python can be loaded with one of the following:

module load Python/3.14.2-GCCcore-15.2.0
module load Python/3.13.5-GCCcore-14.3.0
module load Python/3.13.1-GCCcore-14.2.0
module load Python/3.12.3-GCCcore-13.3.0
module load Python/3.11.5-GCCcore-13.2.0
module load Python/3.11.3-GCCcore-12.3.0
module load Python/3.10.8-GCCcore-12.2.0
module load Python/3.10.8-GCCcore-12.2.0-bare
module load Python/2.7.18-GCCcore-12.2.0-bare

For a standalone environment, the latest module is Python/3.14.2-GCCcore-15.2.0. This loads Python built using GCCcore. Make sure to adjust the module name to match GCC/GCCcore version of any other modules you need to load.

Python virtual environments

Why use a virtual environment?

A venv keeps your project’s additional Python packages separate from other projects and the shared installation. It reuses the module’s Python interpreter, so it does not need to install another Python distribution. Loading compatible package modules also lets you use the cluster-provided packages alongside your own additions.

Create a virtual environment with venv

Create your environment using the same Python module that you will load in future sessions and batch jobs. For module-provided packages such as mpi4py, load the package module first so that its matching Python is used.

Note

--system-site-packages exposes packages installed alongside the module’s Python interpreter. The package selection depends on the loaded module; inspect it with python -m pip list.

Now that the optimised Python interpreter is loaded, you can create a virtual environment using venv.

To keep your virtual environments organised, set a variable in your shell to define a consistent location:

export VENV_HOME="$HOME/.venvs"

Use the same VENV_HOME in each session and in your batch scripts. If you choose the parscratch location below, update the batch examples accordingly.

Note

For larger environments or for heavy users - due to limited home directory storage we recommend building your environments in parscratch:

mkdir -p /mnt/parscratch/users/$USER/
export VENV_HOME="/mnt/parscratch/users/$USER/.venvs"

Let’s create a new venv which will allow you to install user-specific packages while still inheriting system-wide packages.

python -m venv --system-site-packages "$VENV_HOME/my_venv"

  • --system-site-packages This flag allows the virtual environment to use Python module system site packages.

  • my_venv This is the name of your virtual environment. You can replace it with any name you prefer.

After creating the virtual environment, activate it to begin using it:

source "$VENV_HOME/my_venv/bin/activate"

Once activated, you will see (my_venv) appear at the beginning of your command prompt, indicating that you are now working within the virtual environment.

(my_venv) [te1st@node001 [stanage] ~]$

Within the virtual environment, you can install any additional Python packages you need. For instance, to install numpy and pandas:

python -m pip install numpy pandas

These packages will be installed locally in your virtual environment without affecting the system-wide packages. However, we could instead load a SciPy-bundle module, which adds its packages to the PYTHONPATH, making numpy and pandas importable (refer to Packages provided by modules).

To list your virtual environments:

ls "$VENV_HOME"

To exit your virtual environment once you’re done, simply run the following command:

deactivate

Example Using Module System Packages with venv

Load any required package modules before creating your virtual environment, so it uses the matching Python interpreter. For example:

module load mpi4py/4.1.0-gompi-2025b

This module loads Python 3.13.5 and its matching MPI toolchain; use this interpreter rather than Python 3.14.2 for the examples below. The combination of Python packages in venv and modules like mpi4py ensures you have the necessary tools configured for optimal performance and compatibility.

Suppose you are working on a project that requires mpi4py for parallel computing. You can use the modular system loaded Python interpreter along with a venv environment that inherits the optimised MPI configuration:

Since we are demonstrating two approaches — one straightforward and another slightly safer by ensuring Python version consistency — you can choose the option (tab) that best suits your project’s needs.

module load mpi4py/4.1.0-gompi-2025b
python -m venv --system-site-packages "$VENV_HOME/my_mpi_venv"
source "$VENV_HOME/my_mpi_venv/bin/activate"

In this example, mpi4py is already available from the module loaded system installation, enabling you to import mpi4py directly in your Python script. The OpenMPI module is also loaded to ensure proper MPI functionality.

Using this Python venv in a batch job:

Create an example batch submission script called batch.sh that is similar to the following:

#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=1
#SBATCH --time=10:00
#SBATCH --mem=4000

export SLURM_EXPORT_ENV=ALL
module load mpi4py/4.1.0-gompi-2025b
export VENV_HOME="$HOME/.venvs"

# Use the virtual environment you already created
source "$VENV_HOME/my_mpi_venv/bin/activate"

# Run the Python script using srun, which can leverage MPI
srun python my_mpi_work.py

Where import mpi4py is present in my_mpi_work.py, and we use srun so that tasks are properly distributed when using MPI.

Then submit this to Slurm by running:

sbatch batch.sh
Key Points:

  • Use $VENV_HOME for consistent environment location.

  • Append $EBVERSIONPYTHON to environment names to avoid mixing Python versions, whilst also setting $MY_ENV.

Packages provided by modules

These are examples of available modules and the Python packages they provide. Load the required module before creating or activating your virtual environment; these packages are not all included in the standalone Python module.

Module

Python packages

SciPy-bundle/2025.07-gfbf-2025b

numpy, pandas, scipy

Python-bundle-PyPI/2025.07-GCCcore-14.3.0

Includes requests, pytest and pytest-xdist

mpi4py/4.1.0-gompi-2025b

mpi4py

h5py/3.14.0-foss-2025b

h5py

matplotlib/3.10.5-gfbf-2025b

matplotlib

PyYAML/6.0.2-GCCcore-14.3.0

yaml

Pillow/11.3.0-GCCcore-14.3.0

PIL

networkx/3.5-gfbf-2025b

networkx

These module versions use Python 3.13.5. Packages supplied through a module’s PYTHONPATH remain available while that module is loaded; --system-site-packages additionally exposes packages installed alongside the Python interpreter.

With a package module loaded, inspect the packages visible to its Python interpreter with python -m pip list. For example:

module load SciPy-bundle/2025.07-gfbf-2025b
python -m pip list

This module uses Python 3.13.5; use its matching interpreter when creating an environment for these packages.

Attention

Always load a package module that matches the toolchain (i.e., GCC version) as that of the Python module you are using.

Installation notes

Python was installed using Easybuild, build details can be found in $EBROOTPYTHON/easybuild with a given module loaded.

Direct dependencies for latest version of Python