# Failed to run example cases using openmp as backend **URL:** \ **Category:** Errors\ **Created:** [17 July 2022 13:54 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649 "2022-07-17T13:54:24Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [17 July 2022 13:54 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/1 "2022-07-17T13:54:24Z")
Dear all, I am trying to learn PyFR to do some simulations. I have installed PyFR and successfully run the [three examples following the guide](https://pyfr.readthedocs.io/en/latest/examples.html). However, I failed try to run the example case using `openmp` as backend. > **How I installed PyFR** > > OS: Ubuntu 20.04 > GCC version: 9.4.0 > > 1. create a new Python virtual environment by `conda create -n pyfr python=3.10` > 2. install the dependencies by `sudo apt-get install -y libopenmpi-dev openmpi-bin metis libmetis-dev` > 3. install `mpi4py` by `conda install -c conda-forge mpi4py` > 4. install `libxsmm` by `conda install -c conda-forge libxsmm` > 5. install `PyFR` by `pip install pyfr` I try to run the [`inc_cylinder_2d` example case](https://github.com/PyFR/PyFR/tree/develop/examples/inc_cylinder_2d) using `openmp`. The command I used ```auto 1. pyfr import inc_cylinder_2d.msh inc_cylinder_2d.pyfrm 2. pyfr partition 2 inc_cylinder_2d.pyfrm . 3. mpiexec -n 2 pyfr run -b openmp -p inc_cylinder_2d.pyfrm inc_cylinder_2d.ini ``` The error is ```auto 21:23 $ mpiexec -n 2 pyfr run -b openmp -p inc_cylinder_2d.pyfrm inc_cylinder_2d.ini =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 984860 RUNNING AT tang-Workstation = EXIT CODE: 139 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions ``` I also tried to add the following commands to the [original configuration file](https://github.com/PyFR/PyFR/blob/949383f39cfb5f522b975b2b445d917698c4b88c/examples/inc_cylinder_2d/inc_cylinder_2d.ini), but it did not work. ```auto [backend-openmp] cc = gcc gimmik-max-nnz = 100000 ``` Could you give me some suggestions, please? Best regards, Hongwei ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [17 July 2022 17:44 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/2 "2022-07-17T17:44:14Z")
Can you confirm if it works with a single partition? Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [18 July 2022 00:23 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/3 "2022-07-18T00:23:25Z")
Still failed. Does this mean something wrong with the openmpi installation? ```auto $ pyfr run -b openmp -p inc_cylinder_2d.pyfrm inc_cylinder_2d.ini Segmentation fault (core dumped) ``` ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [18 July 2022 09:03 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/4 "2022-07-18T09:03:10Z")
Did you make sure to repartition the case with a single partition before trying this? ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [18 July 2022 09:08 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/5 "2022-07-18T09:08:19Z")
Yes, I can make sure. The full commands I used: ```auto rm inc_cylinder_2d.pyfrm *.pyfrs *.csv pyfr import inc_cylinder_2d.msh inc_cylinder_2d.pyfrm pyfr run -b openmp -p inc_cylinder_2d.pyfrm inc_cylinder_2d.ini ``` ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [18 July 2022 09:10 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/6 "2022-07-18T09:10:10Z")
What is the full error message/traceback? ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [18 July 2022 09:16 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/7 "2022-07-18T09:16:29Z")
Actually, I have done this test on my host machine and using docker (both are ubuntu 20.04). On my host machine, the error is just `Segmentation fault (core dumped)`, nothing more. On the docker, the error is ```auto [b852cfcca23a:80890] ***Process received signal*** [b852cfcca23a:80890] Signal: Segmentation fault (11) [b852cfcca23a:80890] Signal code: Address not mapped (1) [b852cfcca23a:80890] Failing at address: (nil) Segmentation fault (core dumped) ``` I am not sure what caused the difference. On host machine, the `ulimit -u` returns `515044`. On docker container, the `ulimit -u` returns `unlimited` ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [19 July 2022 08:50 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/8 "2022-07-19T08:50:39Z")
So are there no errors from Python? @fdw might have a better idea about what is going on, but to me it seems like either your docker environment is not configured properly or some library/utility is incorrectly compiled. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [19 July 2022 13:10 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/9 "2022-07-19T13:10:34Z")
Can you confirm if any of the mpi4py example programs work? Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [19 July 2022 14:23 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/10 "2022-07-19T14:23:06Z")
Using a clean docker container, I did a simple test using [this example](https://github.com/akkornel/mpi4py). The test result is shown below. Looks fine. ```auto (base) tang@ea1847c729c7:~/mpi4py$ mpirun -n 20 python mpi4.py Controller @ MPI Rank 0: Input 3826176844 Worker at MPI Rank 1: Output 3826176845 is OK (from ea1847c729c7) Worker at MPI Rank 2: Output 3826176846 is OK (from ea1847c729c7) Worker at MPI Rank 3: Output 3826176847 is OK (from ea1847c729c7) Worker at MPI Rank 4: Output 3826176848 is OK (from ea1847c729c7) Worker at MPI Rank 5: Output 3826176849 is OK (from ea1847c729c7) Worker at MPI Rank 6: Output 3826176850 is OK (from ea1847c729c7) Worker at MPI Rank 7: Output 3826176851 is OK (from ea1847c729c7) Worker at MPI Rank 8: Output 3826176852 is OK (from ea1847c729c7) Worker at MPI Rank 9: Output 3826176853 is OK (from ea1847c729c7) Worker at MPI Rank 10: Output 3826176854 is OK (from ea1847c729c7) Worker at MPI Rank 11: Output 3826176855 is OK (from ea1847c729c7) Worker at MPI Rank 12: Output 3826176856 is OK (from ea1847c729c7) Worker at MPI Rank 13: Output 3826176857 is OK (from ea1847c729c7) Worker at MPI Rank 14: Output 3826176858 is OK (from ea1847c729c7) Worker at MPI Rank 15: Output 3826176859 is OK (from ea1847c729c7) Worker at MPI Rank 16: Output 3826176860 is OK (from ea1847c729c7) Worker at MPI Rank 17: Output 3826176861 is OK (from ea1847c729c7) Worker at MPI Rank 18: Output 3826176862 is OK (from ea1847c729c7) Worker at MPI Rank 19: Output 3826176863 is OK (from ea1847c729c7) ``` ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [19 July 2022 16:28 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/11 "2022-07-19T16:28:44Z")
What CPU are you using? Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [20 July 2022 01:52 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/12 "2022-07-20T01:52:24Z")
Well, I think I have found the reason. There is something wrong between `mpi4py` and `Python 3.9+`, see [this issue](https://github.com/mpi4py/mpi4py/issues/57), please. Since `PyFR 1.14.0` requires `Python 3.9+`, I found the only way to install `mpi4py` is via `conda install -c conda-forge mpi4py`. Using `python -m pip install mpi4py` with `Python 3.9+` will finally run into error like [this](https://github.com/mpi4py/mpi4py/issues/57#issuecomment-864530754). However, even though I can successfully install `mpi4py`, the incompatibility between `mpi4py` and `Python 3.9+` could make `PyFR 1.14.0` fail to run when using `openmp` as backend. I have to point out that it may be wrong to say that there is an incompatibility issue between `mpi4py` and `Python 3.9+` since [this example](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/10) is OK. For the details, you can see [here](https://github.com/mpi4py/mpi4py/issues/57#issuecomment-864587905). I have done a test by degrading `PyFR` to `1.10.0` so that I can use `Python 3.8`, and the error disappeared. The example cases can run without error when I use `openmp` as backend. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [20 July 2022 13:31 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/13 "2022-07-20T13:31:10Z")
> There is something wrong between `mpi4py` and `Python 3.9+`, see [this issue](https://github.com/mpi4py/mpi4py/issues/57), please. That issue was closed over a year ago. mpi4py is definitely compatible with Python 3.9. After all, if it was not, then no one would be able to use PyFR 1.14 with multiple GPUs. Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [20 July 2022 14:49 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/14 "2022-07-20T14:49:46Z")
Agree, but I really do not know where it goes wrong. One thing that I can confirm is that the [installation guide](https://pyfr.readthedocs.io/en/latest/installation.html#ubuntu) cannot work on ubuntu 20.04. If you use `pip install pyfr` to install `pyfr`, you will definitely see the `mpi4py` error. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [20 July 2022 14:53 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/15 "2022-07-20T14:53:13Z")
As I asked before, would you be able to confirm what CPU you are using? Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [20 July 2022 14:55 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/16 "2022-07-20T14:55:43Z")
```auto Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz ``` ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [22 July 2022 17:42 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/17 "2022-07-22T17:42:05Z")
How was libxsmm compiled? Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [24 July 2022 00:55 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/18 "2022-07-24T00:55:59Z")
I listed all steps I did for installing PyFR 1.14.0 [here](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649). ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [24 July 2022 14:26 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/19 "2022-07-24T14:26:55Z")
From the install guide the OpenMP backend requires: > [libxsmm](https://github.com/hfp/libxsmm) \>= commit 14b6cea61376653b2712e3eefa72b13c5e76e421 compiled as a shared library (STATIC=0) with BLAS=0 and CODE\_BUF\_MAXSIZE=262144 Note the specific git revision, along with the specific compilation variables on BLAS and CODE\_BUF\_MAXSIZE. There is no reason to believe that the version of libxsmm obtained through `conda` fulfils these requirements. Regards, Freddie. ---
**Author:** ![hwtang](https://avatars.discourse-cdn.com/v4/letter/h/a87d85/32.png) [@hwtang](https://pyfr.discourse.group/u/hwtang)\ **Post date:** [25 July 2022 02:08 UTC](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649/20 "2022-07-25T02:08:22Z")
I have ever tried to build PyFR 1.14.0 from source code following the installation guide. Unfortunately, I still failed to make it. Sorry, I did not save the error information. If I remember correctly, all packages were compiled successfully, but failed to run the cases. [Next page](https://pyfr.discourse.group/t/failed-to-run-example-cases-using-openmp-as-backend/649.md?page=2)