# Running PyFR v2.0.0 in Parallel

**URL:** <https://pyfr.discourse.group/t/running-pyfr-v2-0-0-in-parallel/1066>\
**Category:** Just Starting\
**Created:** [18 March 2024 20:46 UTC](https://pyfr.discourse.group/t/running-pyfr-v2-0-0-in-parallel/1066 "2024-03-18T20:46:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [18 March 2024 20:46 UTC](https://pyfr.discourse.group/t/running-pyfr-v2-0-0-in-parallel/1066/1 "2024-03-18T20:46:22Z")

</div>

Running PyFR v2.0.0 requires several extra dependencies to be correctly configured. Namely _mpi4py_ and a partitioner such as _METIS_ or _SCOTCH_.

## mpi4py

Depending on your platform there is a reasonable chance that mpi4py is broken. A simple test for this is to run:

```auto
$ mpiexec -np 3 python -c 'from mpi4py import MPI; print(MPI.COMM_WORLD.rank)'
0
1
2

```

If you do not see the numbers 0, 1, and 2 output to the screen (in any order) then **your mpi4py package is broken**. The typical cause for this is mpiexec belonging to a different MPI distribution than what mpi4py was compiled against. The solution for this is to recompile mpi4py from scratch (not using a package from conda or pip).

## METIS

Between versions 5.1 and 5.2 METIS broke ABI compatibility. Moreover, the METIS library provides no means of determining its version at runtime. This regrettable—and needless—change on the part of METIS has caused issues for a variety of projects, including PyFR. As it is not possible to support both versions we made the decision with v2.0.0 to only support 5.2.x. This is newer than what most distributions provide. As such if you wish to use METIS it is likely that you will need to compile it from source.

Furthermore, since the 5.1.x series the METIS build scripts **produce a broken shared library**. In particular, `libmetis.so` fails to link against `libGKlib.so` even though it is a dependency. This goes against best practice and has caused issues for numerous projects, including PyFR. The following patch:

> <https://github.com/KarypisLab/METIS/pull/77>

must therefore be applied. If METIS has been built correctly then:

```auto
PYFR_METIS_LIBRARY_PATH=/path/to/your/libmetis.so
python -c "from ctypes import CDLL; CDLL('${PYFR_METIS_LIBRARY_PATH}')"

```

should run without error.

## SCOTCH

In order to make partitioning reproducible PyFR v2.0.0 now depends on SCOTCH v7. This, again, is newer than what most distributions ship. As such, you may also need to compile SCOTCH from source. As with METIS, the SCOTCH build scripts go against best practices and **yield broken shared libraries** by default. Most distributions (including Ubuntu and Gentoo) patch their SCOTCH packages to avoid these issues. On Linux the patchelf:

> **[GitHub - NixOS/patchelf: A small utility to modify the dynamic linker and...](https://github.com/NixOS/patchelf)**
>
> A small utility to modify the dynamic linker and RPATH of ELF executables

tool can be used to add `libscotcherr.so` as a dependency to `libscotch.so` as:

```auto
patchelf --add-needed libscotcherr.so libscotch.so

```

If SCOTCH has been built correctly then:

```auto
PYFR_SCOTCH_LIBRARY_PATH=/path/to/your/libscotch.so
python -c "from ctypes import CDLL; CDLL('${PYFR_SCOTCH_LIBRARY_PATH}')"

```

should run without error.

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [18 March 2024 20:46 UTC](https://pyfr.discourse.group/t/running-pyfr-v2-0-0-in-parallel/1066/2 "2024-03-18T20:46:36Z")

</div>