Attention

The ShARC HPC cluster was decommissioned on the 30th of November 2023 at 17:00. It is no longer possible for users to access that cluster.

Fluent 

ANSYS Fluent is a Computational Fluid Dynamics (CFD) code for modelling fluid flow, heat transfer, mass transfer and chemical reactions. Fluent has interfaces to other pre and post processing packages supplied by ANSYS such as ICEMCFD or Ensight and can incorporate user developed models via user defined functions. Fluent can make use of built in MPI to utilize multiple cross node CPU and can scale to hundreds of cores.

Caution

ANSYS users can now use a maximum of 400 concurrent cores, across all devices on campus, including personal machines and HPC clusters, and regardless of the number of ANSYS applications they have open. Please see ANSYS license restrictions for more detail.

Caution

Do not use versions prior to ANSYS 19.0 if cross-node MPI is taking place as these older versions are no longer supported nor will work correctly with cross-node MPI jobs.

Module loading 

After connecting to ShARC (see Establishing a SSH connection), you can start an interactive graphical session or submit a batch job using ANSYS programs by activating them and making them available with one of the module load commands below:

module load apps/ansys/15.0
module load apps/ansys/16.1
module load apps/ansys/17.2
module load apps/ansys/18.0/binary
module load apps/ansys/18.2/binary
module load apps/ansys/19.0/binary
module load apps/ansys/19.1/binary
module load apps/ansys/19.2/binary
module load apps/ansys/19.3/binary
module load apps/ansys/19.4/binary
module load apps/ansys/20.1/binary
module load apps/ansys/20.2/binary
module load apps/ansys/21.1/binary
module load apps/ansys/21.2/binary
module load apps/ansys/22.2/binary

Interactive jobs 

While using a X11 GUI forwarding supported SSH client, an interactive session can be started on ShARC with the qrshx command which supports graphical applications. You can load an ANSYS module above and then start the fluent program by running the fluent command.

Caution

Warning for ANSYS versions >= 18.0 using the command fluent results in an unresponsive fluent launcher. To launch fluent and bypass the launcher use fluent dim where dim = 2d, 2ddp, 3d or 3ddp or unset the following environment variables before running the command:

unset SGE_TASK_ID
unset RESTARTED

If desired, the ANSYS Workbench GUI executable can be launched with the runwb2 command. To use more than a single core, you should write a batch job script and fluent journal file for submission to the batch queues.

Batch jobs 

To submit Fluent jobs with larger resources a batch job must be submitted to the scheduler. A Fluent batch job is composed of a Fluent journal file which tells Fluent what to do and submission script which tells the scheduler how much resource to request and how to start Fluent.

Fluent Journal files 

A Fluent journal file is effectively a sequence of TUI and/or Scheme commands that you’ll supply to Fluent instead of using the GUI / TUI. A more thorough explanation and tutorial on how to make a Fluent journal file can be found on the following page: Writing Fluent journal files.

Important

It is critical that you consult with the Fluent journal syntax documentation linked above in order to ensure features such as autosaving and intermediate save states are turned on and functioning correctly!

Batch Submission Script 

Fluent is capable of running in both MPI and SMP parallel environments but will use its in-built MPI communications for both. On ShARC, cross process communication must use the RSH protocol instead of SSH. This necessitates the use of either the smp (up to 16 cores on a single node only) or mpi-rsh (as many cores as desired across many nodes) parallel processing environments.

Sample MPI Fluent Scheduler Job Script

The following is an example batch submission script, cfd_job.sh, to run the executable fluent with input journal file test.jou, and carry out a 2D double precision CFD simulation. The script requests 8 cores using the MPI parallel environment mpi-rsh with a runtime of 30 mins and 2 GB of real memory per core. The Fluent input journal file is test.jou.

Hint

The #$ -V SGE option can be used to instruct SGE to import your current terminal environment variables. CAUTION - this may not be desirable and can break job submission if jobs are submitted from an existing interactive job.
Use of the mpi-rsh parallel environment to run MPI parallel jobs for Ansys is required if using more than 16 cores on ShARC.
The 2ddp argument is used to specify a 2D double precision simulation. Valid values include: 2d, 2ddp, 3d and 3ddp
The argument $NSLOTS is a Sun of Grid Engine variable which will return the requested number of cores.
The argument -mpi=intel instructs Fluent to use the in-built Intel MPI communications - important for using the high performance Omnipath networking between nodes.
The argument -pib.infinipath instructs Fluent to use the high performance Omnipath networking. Omnipath networking This will not work on versions prior to 19.0.
The arguments -gu and -driver null instruct Fluent that it will be running with no GUI to avoid errors caused by plot / figure export.
The bash for loop logic is used to construct a machine file to ensure Fluent will allocate MPI tasks correctly.
The -cnf=$MACHINEFILE argument is used to supply the PATH to the machine file created for Fluent.

Caution

The arguments -sgepe mpi-rsh -sge are not required and will cause issues if used with Fluent 21.1 or above.

#!/bin/bash
#$ -cwd
#$ -M a.person@sheffield.ac.uk
#$ -m abe
#$ -l h_rt=00:30:00
#$ -l rmem=2G
#$ -pe mpi-rsh 8
#$ -N JobName

MACHINEFILE="machinefile.$JOB_ID"

for host in `cat $PE_HOSTFILE | awk '{print $1}'`; do
    num=`grep $host $PE_HOSTFILE | awk '{print $2}'`
    for i in `seq 1 $num`; do
    echo $host >> $MACHINEFILE
    done
done

module load apps/ansys/22.2/binary
fluent 3ddp -i test.jou -gu -t $NSLOTS -rsh -mpi=intel -driver null -cnf=$MACHINEFILE -pib.infinipath

Sample SMP Fluent Scheduler Job Script

The following is an example batch submission script, cfd_job.sh, to run the executable fluent with input journal file test.jou, and carry out a 2D double precision CFD simulation. The script requests 8 cores using the SMP parallel environment smp with a runtime of 30 mins and 2 GB of real memory per core. The Fluent input journal file is test.jou. The infinipath option is omitted as this job runs within a single node.

#!/bin/bash
#$ -cwd
#$ -M a.person@sheffield.ac.uk
#$ -m abe
#$ -l h_rt=00:30:00
#$ -l rmem=2G
#$ -pe smp 8
#$ -N JobName

module load apps/ansys/22.2/binary

fluent 2ddp -i test.jou -gu -t $NSLOTS -driver null

Further details about how to construct batch jobs can be found on the batch submission guide page

The job is submitted to the queue by typing:

qsub cfd_job.sh

Exporting GUI Fluent plots in batch submission jobs 

While in a batch job you cannot call a display of a window / figure / animation as Fluent has nowhere to direct this output (there is no available display.)

To avoid this issue while attempting to export images from a batch job you should ensure the fluent command in your submission script has the -gu and -driver null arguments.

i.e. for a ShARC batch job:

fluent 2ddp -i test.jou -gu -t$NSLOTS -rsh -mpi=intel -pib.infinipath -driver null

for a Bessemer batch job:

fluent 2ddp -i test.jou -gu -t$SLURM_NTASKS -driver null

You can setup your plots / graphs either in your journal file or interactively via the GUI and saving the case file.

You can then instruct Fluent to make a hard copy after each figure/animation in the you journal file with:

/display/hard-copy "myfile.tif"

Ensure that you select the correct file format as some file types will not support animations.

For an example of the journal file commands see below:

/display set-window xx ;(xx -> window number)
/display set contours surface yy ;(surface id = yy)
/display set contours filled-contours yes
/display contour zz vof 0 1 ;(contour of volume fraction field (vof) of zz phase, min = 0, max = 1)
/display hard-copy "vof-%t.jpg" ;(save the image file in .jpg format)

Sources 

ANSYS Fluent training and help resources 

Important

Academic support requests should be directed to the IT Services’ Research and Innovation team or the ANSYS Learning Forum (ensure you register with your University email for priority support).

ANSYS provides numerous academic training and help resources including tutorials, video lectures and examples for computational fluid dynamics products. A short list of the resources ANSYS maintains is summarised below:

Notes 

The previous options below have been removed due to a bug in Fluent 21.1 and above which results in incorrect job rewriting and resubmission to the scheduler. All available versions above 18.2 have been tested and should function correctly with the manually specified machine file for batch MPI jobs.

-sge forcing Fluent to recognise job submission via SGE.
-sgepe selecting the mpi-rsh SGE parallel environment.