Even though Open Science Grid is a high throughput computing system, sometimes there is a need to run small OpenMPI based jobs. OSG has limited support for this, as long as the core count is small (4 is known to work well, 8 and 16 becomes more difficult due to the limited number of resources).
To get started, first compile your code using the OpenMPI in the modules system. For example:
$ module load openmpi $ mpicc -o hello hello.c
You can test the executable locally using
$ mpiexec -n 4 hello Hello world from process 1 of 4 Hello world from process 3 of 4 Hello world from process 0 of 4 Hello world from process 2 of 4
To run your code as a job on OSG, first create a
#!/bin/bash set -e module load openmpi mpiexec -n 4 hello
Then, a job submit file:
Requirements = OSGVO_OS_STRING == "RHEL 7" && TARGET.Arch == "X86_64" && HAS_MODULES == True request_cpus = 4 request_memory = 4 GB executable = wrapper.sh transfer_input_files = hello output = job.out.$(Cluster).$(Process) error = job.error.$(Cluster).$(Process) log = job.log.$(Cluster).$(Process) on_exit_hold = (ExitBySignal == True) || (ExitCode != 0) PeriodicRelease = ( (CurrentTime - EnteredCurrentStatus) > 300 ) && (NumJobStarts < 5) queue 1
Note how the executable is the
wrapper.sh script, and that the real executable
transferred using the
Please make sure that the number of cores specified in the submit file via
request_cpus match the
-n argument in the
This page was updated on Jul 23, 2019 at 11:01 from start/resources/openmpi-jobs.md.