# How to set openmpi

**URL:** <https://pyfr.discourse.group/t/how-to-set-openmpi/777>\
**Category:** Cases\
**Tags:** hpc\
**Created:** [18 March 2023 13:12 UTC](https://pyfr.discourse.group/t/how-to-set-openmpi/777 "2023-03-18T13:12:11Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![fei](https://avatars.discourse-cdn.com/v4/letter/f/59ef9b/32.png) [@fei](https://pyfr.discourse.group/u/fei)\
**Post date:** [18 March 2023 13:12 UTC](https://pyfr.discourse.group/t/how-to-set-openmpi/777/1 "2023-03-18T13:12:11Z")

</div>

Hello,

First computer with one socket and 12 cores, I run with “pyfr run -b openmp -p \*.pyfrm \*.ini”, the calculation time is approximately 130000.

Second computer with two sockets and 32 cores, as the performance tuning guide said, on a two socket system it is suggested to run PyFR with two MPI ranks, with each process being bound to a single socket. Then I run with “mpiexec -n 2 pyfr run -b openmp -p \*.pyfrm \*.ini”, the calculation time is approximately 500000. And I run with “pyfr run -b openmp -p \*.pyfrm \*.ini” , the calculation time is approximately 200000, which is more than the first computer.

The backend-openmp is attached.  
 ![image](https://global.discourse-cdn.com/free1/uploads/pyfr/original/1X/82af7eead4028870831507273e1b8dbda28986b4.png)

Is there anything else I need to optimize?

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [18 March 2023 14:07 UTC](https://pyfr.discourse.group/t/how-to-set-openmpi/777/2 "2023-03-18T14:07:38Z")

</div>

You need to tell OpenMPI to assign a certain number of CPU cores to each rank. Further, you will also want to explicitly set the number of OpenMP threads. This is all described in the OpenMPI manual. Finally, you will also want to make sure each socket is a single NUMA zone. The `lscpu` command can let you know if this is the case.

Failing all of that, you can partition into 32 pieces and then just run with 32 ranks and `OMP_NUM_THREADS=1`. The performance will not be as good, but it is easier to set up.

Regards, Freddie.