Python

serial python

The following python script (hello_world.py) is a simple script.

hello_world.py

print ("Hello, World!")

submit.sh

#!/bin/bash
#SBATCH --job-name=serial_python
#SBATCH --output=serial_python.out
#SBATCH --partition=standard

# execute Python script
srun python3 hello_world.py

Data Parallelism in Python

The multiprocessing module in Python provides users the ability to distribute multiple inputs to a single function over multiple processors (data parallelism). This is useful when evaluating a single function multiple times so that the computations are shared by multiple processes.

Python script

The following example is a demonstration of a simple function being distributed to multiple processor cores.

import os
import time
import multiprocessing as mproc

def myfunc(x):

    # 1 second delay
    time.sleep(1)
    xsq = x**2

    # get information on the current process and print to stdout
    proc = mproc.current_process()
    print ('{0:2}^2 = {1:4} calculated on {2} (pid {3})'.format(x, xsq, proc.name, proc.pid), flush=True)

    return xsq


#------------------------------------------------------------------------------
# USE A TIMER FOR TESTING SERIAL EXECUTION

# start timer
start = time.time()

# serial processes
result_serial = [myfunc(i) for i in range(8)]

# stop timer and calculate elapsed time
stop = time.time()
time_elapsed = stop - start
print ('\n Serial job completed in {:.2f} seconds\n'.format(time_elapsed))

#------------------------------------------------------------------------------
# USE A TIMER FOR TESTING PARALLEL EXECUTION

# get the number of processors directly from the submit.sh script
pool_size = int(os.environ['SLURM_JOB_CPUS_PER_NODE'])

# start timer
start = time.time()

# data parallel execution -- "with" statement closes pool when done
with mproc.Pool(processes=pool_size) as pool:
    result_parallel = pool.map(myfunc, range(8))

# stop timer and calculate elapsed time
stop = time.time()
time_elapsed = stop - start
print ('\n Parallel job ({} CPUs) completed in {:.2f} seconds\n'.format(pool_size, time_elapsed))

#------------------------------------------------------------------------------
# VERIFY THAT SERIAL AND PARALLEL METHODS PRODUCE SAME RESULTS

# check that serial and parallel results are equivalent
if (result_serial == result_parallel):
    print ('\n Results from serial and parallel calculation are equal!\n')

Submission script

To submit the job for parallel execution on Mercury, use a submission script similar to the following:

#!/bin/bash

#SBATCH --cpus-per-task=4      # launch 4 CPUs (cores)  per task
#SBATCH --job-name=smp-py      # name of job
#SBATCH --partition=standard   # assign the job to a specific queue
#SBATCH --output=smp-py.log    # join the output and error files

# load desired python version (check availability with 'module avail')
module load python/booth/3.8

# execute python script
srun python3 mp.py

In submit.sh, line #3 requests four CPU cores to be made available for the task. This job has only 1 task. You can scale up to the Max Cores value listed in the Partitions and Limits table, however we generally recommend 12 as a high value. Parallel efficiency drops as core count increases, thus high core counts should be justified by profiling your parallelism efficiency. Additionally, high core count jobs will generally be queued longer while awaiting resources to become available. The name of the job on line #4 is user-specified and should be a descriptive name subject to the following rule:

The name may be any arbitrary alphanumeric ASCII string, but
may not contain  "\n", "\t", "\r", "/", ":", "@", "\", "*",
or "?".

Line #5 assigns the job to a specific queue (for more info, see Queues and Limits). Line #10 loads the requested software in the user’s environment. More information on modules can be found in Software Modules. Job Submission

Finally, to submit the job from a login node on Mercury, simply type sbatch submit.sh at the prompt. Note that both files (mp.py and submit.sh) should be in the same directory. Sample Output Running this example produces the following output:

0^2 =    0 calculated on MainProcess (pid 30017)
1^2 =    1 calculated on MainProcess (pid 30017)
2^2 =    4 calculated on MainProcess (pid 30017)
3^2 =    9 calculated on MainProcess (pid 30017)
4^2 =   16 calculated on MainProcess (pid 30017)
5^2 =   25 calculated on MainProcess (pid 30017)
6^2 =   36 calculated on MainProcess (pid 30017)
7^2 =   49 calculated on MainProcess (pid 30017)

Serial job completed in 8.02 seconds


0^2 =    0 calculated on ForkPoolWorker-1 (pid 30019)
3^2 =    9 calculated on ForkPoolWorker-4 (pid 30022)
2^2 =    4 calculated on ForkPoolWorker-3 (pid 30021)
1^2 =    1 calculated on ForkPoolWorker-2 (pid 30020)
4^2 =   16 calculated on ForkPoolWorker-1 (pid 30019)
5^2 =   25 calculated on ForkPoolWorker-4 (pid 30022)
6^2 =   36 calculated on ForkPoolWorker-3 (pid 30021)
7^2 =   49 calculated on ForkPoolWorker-2 (pid 30020)

Parallel job (4 CPUs) completed in 2.10 seconds


Results from serial and parallel calculation are equal!

gpu python

how to

Virtual Environment

Virtual environments are a convenient way to manage Python environments. You can reuse virtual environments and modify them by installing or removing components at any time.

# start an interactive session
[mfe01] $ srun --pty bash --login

# load the python module of your choice
[mcn01] $ module load python/booth/3.8

# create a new virtual environment (here named "myenv")
[mcn01] $ virtualenv --system-site-packages -p python3 ~/venv/myenv

# activate the virtual environment
[mcn01] $ source ~/venv/myenv/bin/activate

# install packages from pip
[mcn01] (myenv) $ pip install <python-package>

# when finished, leave the venv
[mcn01] (myenv) $ deactivate

# log out of the compute node
[mcn01] $ exit

Python packages installed in a virtual environment are available only inside the virtual environment. A submit script to activate the virtual environment called “myenv” may look like the following:

#!/bin/bash

#SBATCH --account=<account-name>

# load the python module
module load python/booth/3.8

# activate the venv
source ~/venv/myenv/bin/activate

# load package that was installed in the venv
srun python3 -c "import <python-package>"