OpenFOAM

Guidance last updated: 22 October 2026

OpenFOAM is free, open-source software for computational fluid dynamics (CFD). Different OpenFOAM distributions exist; the module below provides the OpenCFD release v2306.

Documentation: https://www.openfoam.com/documentation/overview

For the OpenFOAM Foundation User Guide, see https://cfd.direct/openfoam/user-guide.

Usage

After connecting to Stanage (see Establishing a SSH connection), start an interactive session:

salloc

Load OpenFOAM with one of the following:

module load OpenFOAM/v2306-foss-2022b

OpenFOAM can be used in an interactive session or a batch job. OpenFOAM modules can be activated using the module file and sourcing the OpenFOAM environment script e.g.

module load OpenFOAM/v2306-foss-2022b
source "$FOAM_BASH"

Hint

Check the documentation for each OpenFOAM executable to determine whether it supports parallel execution. In the examples below, blockMesh, decomposePar and reconstructPar run serially; simpleFoam runs through srun with -parallel in the MPI job.

Interactive Usage

In an interactive session, the following example runs the supplied pitzDaily case. Choose a new working directory so existing results are preserved:

module load OpenFOAM/v2306-foss-2022b
source "$FOAM_BASH"
mkdir "$HOME/openfoam-pitzDaily-serial"
cd "$HOME/openfoam-pitzDaily-serial"
cp -r "$FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily" .
cd pitzDaily
blockMesh
simpleFoam

Batch Usage

The following example batch submission script runs pitzDaily with four MPI ranks across two nodes, with 1000 MB per CPU (2000 MB per node). Save the decomposeParDict shown below alongside this script and submit it with sbatch my_job.sh. The working directory must be accessible from both nodes.

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=2
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1000
#SBATCH --time=00:15:00

set -e
export SLURM_EXPORT_ENV=ALL
module load OpenFOAM/v2306-foss-2022b
source "$FOAM_BASH"

mkdir "pitzDaily-$SLURM_JOB_ID"
cp -r "$FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily/." "pitzDaily-$SLURM_JOB_ID/"
cp decomposeParDict "pitzDaily-$SLURM_JOB_ID/system/decomposeParDict"
cd "pitzDaily-$SLURM_JOB_ID"

blockMesh
decomposePar
srun simpleFoam -parallel
reconstructPar

Example decomposeParDict

numberOfSubdomains must match the total MPI rank count. Save this example as decomposeParDict in the submission directory. The simpleCoeffs n values specify the divisions in each direction; their product must match numberOfSubdomains. Other decomposition methods are described in the OpenFOAM documentation.

FoamFile
{
    version 2.0;
    format ascii;
    class dictionary;
    object decomposeParDict;
}

numberOfSubdomains 4;
method simple;
simpleCoeffs
{
    n (2 2 1);
    delta 0.001;
}

Attention

OpenFOAM can create many small files on the Fastdata (Lustre) area. Please follow the instructions below to use collated file handlers to reduce I/O overhead.

OpenFOAM I/O on Fastdata (Lustre)

Uncollated output

By default, each process in an OpenFOAM parallel simulation writes one file for each output field at each output time. Large numbers of small files increase filesystem metadata operations and can slow down your job and other users’ jobs on the Fastdata (Lustre) area (/mnt/parscratch).

Collated file handler

The concept of “collating” OpenFOAM output is to merge the results of a variable corresponding to a group of processors (ranks) into a single file. File count then depends on the number of I/O groups rather than the number of ranks.

For example, 64 MPI ranks writing 20 fields at 100 output times produce the following approximate field-file counts. These exclude mesh and other files.

Field-file counts with and without collating
Output arrangement	Calculation	Field files
Separate files per rank	64 × 20 × 100	128,000
Four I/O groups of 16 ranks	4 × 20 × 100	8,000

In OpenCFD v2306, hostCollated groups ranks by host by default. To choose the groups explicitly, set FOAM_IORANKS to the first rank in each group. Every I/O rank must be less than the total rank count, and the list must start at zero.

Example: four I/O groups with 64 ranks

This example uses 64 ranks, rather than the four ranks in the batch script above. For a prepared simpleFoam case with numberOfSubdomains 64 and a matching decomposition in system/decomposeParDict, select four groups of 16 ranks with:

export FOAM_IORANKS='(0 16 32 48)'

Use the same file handler and I/O rank settings when decomposing the case, running the solver and reconstructing results. Within a 64-rank allocation, run:

decomposePar -fileHandler hostCollated
srun --ntasks=64 simpleFoam -parallel -fileHandler hostCollated
reconstructPar -fileHandler hostCollated

The decomposePar command above creates the grouped processor directories below; the solver then writes each group’s field data into its directory:

processors64_0-15
processors64_16-31
processors64_32-47
processors64_48-63

See the OpenFOAM hostCollated documentation for further details.

Other ways to reduce I/O

If reading and writing files takes up a significant fraction of your job’s time, you can modify the input and output settings in system/controlDict. Some of these suggestions may not be feasible depending on your analysis needs:

Increase writeInterval to write less frequently and produce fewer output directories for the same simulated period. Its meaning depends on writeControl.
Use writeFormat binary to reduce field-file size compared with ASCII.
For steady-state calculations where only the latest result is needed, purgeWrite 1 retains only the latest automatically written time directory.
Set runTimeModifiable no if you do not need OpenFOAM to reread modified dictionaries during the calculation.

Installation notes

OpenFOAM was installed using Easybuild, build details can be found in $EBROOTOPENFOAM/easybuild with a given module loaded.