Shared Memory Parallelism (SMP)¶

OpenMP¶

OpenMP (Open Multi-Processing) is one of the most common programming frameworks used to implement shared-memory parallelism. It greatly simplifies the tasks of distributing work and coordination in a multi-threaded C, C++ or Fortran program. Many research applications make use of the OpenMP library: you can often enable OpenMP support at compile-time and/or run-time using compiler flags, command-line options and/or configuration files.

Batch OpenMP jobs¶

To submit a SMP job to our system, we we need to request a particular Parallel Environment in our batch job submission scripts. On ShARC your script needs to include a line like:

#$ -pe smp N

whereas on Iceberg it needs to be:

#$ -pe openmp N

Change N to the number of processor cores you wish to use.

Here’s an example that requests 4 cores on ShARC:

#!/bin/bash
# Request 4 cores from ShARC's scheduler
#$ -pe smp 4
#$ Request 4 GB of memory per CPU core
#$ -l rmem=4G

# Tell programs that use the OpenMP library to use 4 cores
export OMP_NUM_THREADS=4
./myprogram

Note that you have to specify the number of cores at least twice when running OpenMP jobs:

• Once to the scheduler (#$ -pe smp 4) so that it knows how many cores to assign to your job;

• Once to the OpenMP runtime environment (export OMP_NUM_THREADS=4) so that OpenMP knows how many threads to create.

• You may also need to expliclty need to tell your application how many threads it should create.

Running other SMP schemes¶

Any system that uses shared memory parallelism, such as pthreads, Python’s multiprocessing module, R’s mcapply and so on can be submitted using the smp (ShARC) / openmp (Iceberg) Parallel Environments. Many of these also check the OMP_NUM_THREADS environment variable to determine how many threads to create.

Interactive SMP jobs¶

You can start an interactive SMP job on ShARC like so:

qrsh -pe smp 4 -l rmem=3G

Or like so on Iceberg:

qrsh -pe openmp 4 -l rmem=3G