# KeyError on 3D case running with WSL Ubuntu openMP

**URL:** <https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404>\
**Category:** Errors\
**Tags:** windows, openmp\
**Created:** [27 May 2021 04:26 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404 "2021-05-27T04:26:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [27 May 2021 04:26 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/1 "2021-05-27T04:26:26Z")

</div>

Hi All,

I am running PyFr on Windows WSL Ubuntu with openMP. The 2D example cases work well.

However, when I prepare my own 3D case and tried with the following command: _mpiexec -n 2 pyfr run -b openmp -p building\_3d.pyfrm building\_3d.ini_, I am getting a key error.

Would anyone know what cause this error? Is it caused by unsupported environment / improper installation or divergence issue on the simulation? I would appreciate very much if anyone can shed some light on this issue.

Thanks,  
Jack

```auto
Traceback (most recent call last):
  File "/home/user/ENV3/bin/pyfr", line 8, in <module>
    sys.exit(main())
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/ __main__.py", line 117, in main
    args.process(args)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/ __main__.py", line 245, in process_run
    _process_common(
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/ __main__.py", line 227, in _process_common
    solver = get_solver(backend, rallocs, mesh, soln, cfg)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/solvers/ __init__.py", line 16, in get_solver
    return get_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/ __init__.py", line 36, in get_integrator
    return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/phys/controllers.py", line 8, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/phys/base.py", line 17, in __init__
    self.pseudointegrator = get_pseudo_integrator(
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/ __init__.py", line 58, in get_pseudo_integrator
    return DualMultiPIntegrator(backend, systemcls, rallocs, mesh,
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/multip.py", line 123, in __init__
    self.pintgs[l] = lpsint(backend, systemcls, rallocs, mesh,
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/pseudocontrollers.py", line 90, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/pseudocontrollers.py", line 12, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/pseudosteppers.py", line 191, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/integrators/dual/pseudo/base.py", line 50, in __init__
    self.system = systemcls(backend, rallocs, mesh, initsoln,
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/solvers/base/system.py", line 67, in __init__
    self._gen_kernels(eles, int_inters, mpi_inters, bc_inters)
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/solvers/base/system.py", line 188, in _gen_kernels
    kernels[pn, kn].append(kgetter())
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/solvers/baseadvec/elements.py", line 63, in <lambda>
    kernels['tdivtpcorf'] = lambda: self._be.kernel(
  File "/home/user/ENV3/lib/python3.8/site-packages/pyfr/backends/base/backend.py", line 168, in kernel
    raise KeyError(f'Kernel "{name}" has no providers')
KeyError: 'Kernel "mul" has no providers'

```

---

<div class="post-metadata">

**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\
**Post date:** [27 May 2021 15:17 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/2 "2021-05-27T15:17:02Z")

</div>

Hi Jack, wlecome to the forum. Would you be able to post your ini file?

---

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [27 May 2021 19:25 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/3 "2021-05-27T19:25:37Z")

</div>

Hi Will, below is my ini file: [.ini - Google Drive](https://drive.google.com/file/d/1MHLbB-gU71z8nXMcbApyuHuS0hh3E5jZ/view?usp=sharing)

Let me also post the .msh file here: [.msh - Google Drive](https://drive.google.com/file/d/1c8SE-KqSTOHu0M9euniSAFJbsxdJFUFF/view?usp=sharing)

```auto
[backend]
precision = double
rank-allocator = linear

[constants]
nu = 0.005
Uin = 1.0
Vin = 0.0
Win = 0.0
Pc = 1.0
ac-zeta = 2.5

[solver]
system = ac-navier-stokes
order = 3

[solver-time-integrator]
formulation = dual
scheme = bdf2
pseudo-scheme = rk34
controller = none
pseudo-controller = local-pi
tstart = 0.0
tend = 75.0
dt = 0.0125
pseudo-dt = 0.0025
pseudo-niters-min = 3
pseudo-niters-max = 3
pseudo-resid-norm = l2
pseudo-resid-tol = 1e-12
atol = 1e-6
pseudo-dt-max-mult = 2.5

[solver-dual-time-integrator-multip]
pseudo-dt-fact = 1.75
cycle = [(3, 1), (2, 1), (1, 1), (0, 2), (1, 1), (2, 1), (3, 4)]

[solver-interfaces]
riemann-solver = rusanov
ldg-beta = 0.5
ldg-tau = 0.1

[solver-interfaces-line]
flux-pts = gauss-legendre

[solver-interfaces-quad]
flux-pts = gauss-legendre

[solver-elements-tri]
soln-pts = williams-shunn

[solver-elements-quad]
soln-pts = gauss-legendre

[solver-elements-hex]
soln-pts = gauss-legendre

[soln-plugin-nancheck]
nsteps = 50

[soln-plugin-pseudostats]
flushsteps = 20
file = residual.csv
header = true

[soln-plugin-writer]
dt-out = 5.0
basedir = .
basename = inc_cylinder_2d_{t:.2f}

[soln-bcs-ground]
type = no-slp-wall

[soln-bcs-building]
type = no-slp-wall

[soln-bcs-inlet]
type = ac-char-riem-inv
ac-zeta = 180
p = Pc
u = Uin
v = Vin
w = Win

[soln-bcs-outlet]
type = ac-char-riem-inv
ac-zeta = 180
p = Pc
u = Uin
v = Vin
w = Win

[soln-bcs-top]
type = no-slp-wall

[soln-bcs-right]
type = no-slp-wall

[soln-ics]
u = Uin
v = Vin
w = Win
p = Pc

[backend-openmp]
cc = gcc 
cflags = -L/usr/lib -llapack -lblas

```

Thanks,  
Jack

---

<div class="post-metadata">

**Author:** ![tdzanic](https://avatars.discourse-cdn.com/v4/letter/t/f05b48/32.png) [@tdzanic](https://pyfr.discourse.group/u/tdzanic)\
**Post date:** [27 May 2021 20:10 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/4 "2021-05-27T20:10:18Z")

</div>

Hi Jack,

Have you tried directly providing the path to libblas in the backend-openmp section? Something like:

cblas= example/path/libblas.so

---

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [27 May 2021 21:17 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/5 "2021-05-27T21:17:22Z")

</div>

Hi tdzanic, thanks for looking into this. I tried that and I am still getting the same error.

---

<div class="post-metadata">

**Author:** ![tdzanic](https://avatars.discourse-cdn.com/v4/letter/t/f05b48/32.png) [@tdzanic](https://pyfr.discourse.group/u/tdzanic)\
**Post date:** [27 May 2021 21:34 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/6 "2021-05-27T21:34:16Z")

</div>

Does the same issue come up when you run serially, or is it only when you try to run in parallel?

---

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [27 May 2021 21:48 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/7 "2021-05-27T21:48:36Z")

</div>

Hi tdzanic,

I just tried running:  
pyfr partition 1 building\_3d.pyfrm .  
pyfr run -b openmp -p building\_3d.pyfrm building\_3d.ini

But I am still getting the same error.

Jack

---

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [27 May 2021 23:20 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/8 "2021-05-27T23:20:59Z")

</div>

I am running some diagnostics on the code. It seems that it is trying to find the provider for “mul”.  
It could only find OpenMPGiMMiKKernels.mul, but it is giving NotSuitableError: Matrix too dense for GiMMiK. The number of non-zero terms in the matrix is 768 which is larger than the 512 limit.

Should there be another provider for dense matrix multiplication?

```auto
Starting: <pyfr.backends.openmp.provider.OpenMPPointwiseKernelProvider object at 0x7fc629cc53a0> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_arglst', '_build_kernel', '_instantiate_kernel', '_render_kernel', 'backend', 'bccflux', 'bcconu', 'gradcoru', 'gradcorulin', 'intcflux', 'intconu', 'kernel_generator_cls', 'negdivconf', 'register', 'tflux', 'tfluxlin']
Getting: None
Starting: <pyfr.backends.openmp.blasext.OpenMPBlasExtKernels object at 0x7fc629cc5430> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', 'axnpby', 'backend', 'copy', 'errest']
Getting: None
Starting: <pyfr.backends.openmp.packing.OpenMPPackingKernels object at 0x7fc629cc5460> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', '_sendrecv', 'backend', 'pack', 'recv_pack', 'send_pack', 'unpack']
Getting: None
Starting: <pyfr.backends.openmp.gimmik.OpenMPGiMMiKKernels object at 0x7fc629cc5370> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', 'backend', 'max_nnz', 'mul']
Getting: <bound method OpenMPGiMMiKKernels.mul of <pyfr.backends.openmp.gimmik.OpenMPGiMMiKKernels object at 0x7fc629cc5370>>
(<pyfr.backends.openmp.types.OpenMPConstMatrix object at 0x7fc617e49100>, <pyfr.backends.openmp.types.OpenMPMatrixBank object at 0x7fc629cefbb0>) {'out': <pyfr.backends.openmp.types.OpenMPMatrix object at 0x7fc629cef790>}
Starting: <pyfr.backends.openmp.provider.OpenMPPointwiseKernelProvider object at 0x7fc629cc53a0> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_arglst', '_build_kernel', '_instantiate_kernel', '_render_kernel', 'backend', 'bccflux', 'bcconu', 'gradcoru', 'gradcorulin', 'intcflux', 'intconu', 'kernel_generator_cls', 'negdivconf', 'register', 'tflux', 'tfluxlin']
Getting: None
Starting: <pyfr.backends.openmp.blasext.OpenMPBlasExtKernels object at 0x7fc629cc5430> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', 'axnpby', 'backend', 'copy', 'errest']
Getting: None
Starting: <pyfr.backends.openmp.packing.OpenMPPackingKernels object at 0x7fc629cc5460> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', '_sendrecv', 'backend', 'pack', 'recv_pack', 'send_pack', 'unpack']
Getting: None
Starting: <pyfr.backends.openmp.gimmik.OpenMPGiMMiKKernels object at 0x7fc629cc5370> mul
Starting: [' __class__', ' __delattr__', ' __dict__', ' __dir__', ' __doc__', ' __eq__', ' __format__', ' __ge__', ' __getattribute__', ' __gt__', ' __hash__', ' __init__', ' __init_subclass__', ' __le__', ' __lt__', ' __module__', ' __ne__', ' __new__', ' __reduce__', ' __reduce_ex__', ' __repr__', ' __setattr__', ' __sizeof__', ' __str__', ' __subclasshook__', ' __weakref__', '_build_kernel', '_memoize_cache', 'backend', 'max_nnz', 'mul']
Getting: <bound method OpenMPGiMMiKKernels.mul of <pyfr.backends.openmp.gimmik.OpenMPGiMMiKKernels object at 0x7fc629cc5370>>
(<pyfr.backends.openmp.types.OpenMPConstMatrix object at 0x7fc608b20b20>, <pyfr.backends.openmp.types.OpenMPMatrix object at 0x7fc629cefc10>) {'out': <pyfr.backends.openmp.types.OpenMPMatrixBank object at 0x7fc629cefc40>}
NotSuitableError: Matrix too dense for GiMMiK

```

---

<div class="post-metadata">

**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\
**Post date:** [28 May 2021 03:26 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/9 "2021-05-28T03:26:42Z")

</div>

At least for pyfr v1.12 you can manually set the cutoff number of non-zeros for gimmik with `gimmik-max-nnz = 512` in the openmp backend section. Although this will have an effect on performance. If you have a look at the gimmik paper you can see how the performance changes with matrix size and sparsity.

The libxsmm library can also be an option, see the [PyFR OpenMP documentation](https://pyfr.readthedocs.io/en/latest/user_guide.html#backend-openmp) and the [Libxsmm git](https://github.com/hfp/libxsmm).

PyFR will also fall back on a blas gemm if the operators are too dense or too big. You may want to check that you have a blas library in your library path. On WSL I think you can do `sudo apt install libblas-dev` for a quick install, or you can compile your own, or if you have it Intel’s MKL can be a more performant option.

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [28 May 2021 18:54 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/10 "2021-05-28T18:54:03Z")

</div>

There are two options here. The first is to increase `gimmik-max-nnz` as:

```auto
[backend-openmp]
gimmik-max-nnz = 8192

```

This should work, although performance may be far from optimal. Peak performance can be had through installing libxsmm. Due to bugs you will need to compile the very latest version from GitHub:

> **[hfp/libxsmm](https://github.com/hfp/libxsmm)**
>
> Library for specialized dense and sparse matrix operations, and deep learning primitives. - hfp/libxsmm

Be sure to compile the library with `make SHARED=1 NOBLAS=1`. Then either add `libxsmm.so` to your library path or `export PYFR_XSMM_LIBRARY_PATH=/path/to/compiled/libxsmm.so`

Support for CBLAS (MKL, OpenBLAS, etc) was removed as of v1.12 due to a change in the underlying memory layout.

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![jack](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jack](https://pyfr.discourse.group/u/jack)\
**Post date:** [28 May 2021 19:35 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/11 "2021-05-28T19:35:43Z")

</div>

Thanks! I tried increasing gimmik-max-nnz and it worked.

Jack

---

<div class="post-metadata">

**Author:** ![Zhenyang](https://avatars.discourse-cdn.com/v4/letter/z/ec9cab/32.png) [@Zhenyang](https://pyfr.discourse.group/u/Zhenyang)\
**Post date:** [2 September 2021 18:01 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/12 "2021-09-02T18:01:43Z")

</div>

Hi Freddie,

I met the same problem while running cases with openmp backend. I tried increasing the gimmik-max-nnz number but only works for smaller cases. I also tried to install libxsmm but still not working. Even running example case I have the very same problem without setting gimmik parameters which could work normally before. Any idea about what has happened? Thanks ahead!

Best wishes,  
Zhenyang

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [2 September 2021 18:29 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/13 "2021-09-02T18:29:49Z")

</div>

We are gradually migrating the OpenMP backend to requiring libxsmm, and this includes poorly performing fallbacks (cblas, and in the future GiMMiK). Nothing else is competitive in terms of performance.

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![Zhenyang](https://avatars.discourse-cdn.com/v4/letter/z/ec9cab/32.png) [@Zhenyang](https://pyfr.discourse.group/u/Zhenyang)\
**Post date:** [3 September 2021 19:17 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/14 "2021-09-03T19:17:59Z")

</div>

Hi Freddie,

Thanks for reply. Actually I have compiled the libxsmm library but it is still not working. I have added the shared library to my library path. The same error is still there.

Best wishes,  
Zhenyang

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [3 September 2021 19:47 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/15 "2021-09-03T19:47:10Z")

</div>

If you have added the library correctly the following command should run without error:

```auto
python -c "from ctypes import CDLL; CDLL('libxsmm.so')"

```

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![Zhenyang](https://avatars.discourse-cdn.com/v4/letter/z/ec9cab/32.png) [@Zhenyang](https://pyfr.discourse.group/u/Zhenyang)\
**Post date:** [3 September 2021 20:21 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/16 "2021-09-03T20:21:31Z")

</div>

Hi Freddie,

Yes that command can run without any error. Here is what I got:

```auto
zhenyang@blefjell:/scratch/zhenyang/compute/pyfr/channel/channel_local/run$ python -c "from ctypes import CDLL; CDLL('libxsmm.so')"
zhenyang@blefjell:/scratch/zhenyang/compute/pyfr/channel/channel_local/run$ mpirun -np 12 pyfr run -b openmp -p mesh_channel.pyfrm channel_Retau180.ini 
Traceback (most recent call last):
  File "/home/mech/zhenyang/.local/bin/pyfr", line 8, in <module>
    sys.exit(main())
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/ __main__.py", line 117, in main
    args.process(args)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/ __main__.py", line 246, in process_run
    args, NativeReader(args.mesh), None, Inifile.load(args.cfg)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/ __main__.py", line 227, in _process_common
    solver = get_solver(backend, rallocs, mesh, soln, cfg)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/solvers/ __init__.py", line 16, in get_solver
    return get_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/integrators/ __init__.py", line 36, in get_integrator
    return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/integrators/std/controllers.py", line 86, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/integrators/std/controllers.py", line 14, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/integrators/std/steppers.py", line 133, in __init__
    super(). __init__ (*args, **kwargs)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/integrators/std/base.py", line 28, in __init__
    nregs=self.nregs, cfg=cfg)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/solvers/base/system.py", line 67, in __init__
    self._gen_kernels(eles, int_inters, mpi_inters, bc_inters)
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/solvers/base/system.py", line 188, in _gen_kernels
    kernels[pn, kn].append(kgetter())
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/solvers/baseadvec/elements.py", line 53, in <lambda>
    out=self._scal_fpts
  File "/home/mech/zhenyang/.local/lib/python3.7/site-packages/pyfr/backends/base/backend.py", line 167, in kernel
    raise KeyError(f'Kernel "{name}" has no providers')
KeyError: 'Kernel "mul" has no providers'
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

```

Best wishes,  
Zhenyang

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [3 September 2021 20:54 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/17 "2021-09-03T20:54:05Z")

</div>

Can you post your config file?

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![Zhenyang](https://avatars.discourse-cdn.com/v4/letter/z/ec9cab/32.png) [@Zhenyang](https://pyfr.discourse.group/u/Zhenyang)\
**Post date:** [3 September 2021 21:02 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/18 "2021-09-03T21:02:13Z")

</div>

Hi Freddie,

here is ini file

```auto
[backend]
precision = double
rank-allocator = linear

;[backend-cuda]
;device-id = local-rank
;mpi-type = standard
;block-1d = 64
;block-2d = 128

[backend-openmp]
cc = gcc
cblas = /usr/lib/x86_64-linux-gnu/atlas/libblas.so. ; Normally I dont need this line; but i tried both are not working
;gimmik-max-nnz = 8192

[constants]
cInf = 1.0
rhoInf = 1.0
Mach = 0.1
Re = 2767
l = 1.0
gamma = 1.4

uInf = 0.1 ; Mach*cInf
vInf = 0.0
wInf = 0.0
pInf = 0.714285714 ; cInf*cInf*rhoInf/gamma
EInf = 1.7907142849999997 ; pInf/(Gamma-1)+0.5*rhoInf*(uInf*uInf+vInf*vInf)
mu = 0.00003614022406938923 ; uInf*l*rhoInf/Re
Pr = 0.72

[solver]
system = navier-stokes
order = 4

[solver-time-integrator]
scheme = rk45
controller = pi
tstart = 0.0
tend = 100.0

atol = 0.000001
rtol = 0.001
errest-norm = l2
safety-fact = 0.9
min-fact = 0.3
max-fact = 2.5
dt = 0.0001

[solver-interfaces]
riemann-solver = roem
ldg-beta = 0.5
ldg-tau = 0.1

[solver-interfaces-quad]
flux-pts = gauss-legendre
quad-deg = 9
quad-pts = gauss-legendre

[solver-interfaces-line]
flux-pts = gauss-legendre
quad-deg = 9
quad-pts = gauss-legendre

[solver-elements-tri]
soln-pts = williams-shunn
quad-deg = 9

[solver-elements-quad]
soln-pts = gauss-legendre
quad-deg = 9

[solver-elements-hex]
soln-pts = gauss-legendre
quad-deg = 9
quad-pts = gauss-legendre

[soln-plugin-writer]
dt-out = 1
basedir = .
basename = channel_{t:.2f}

[soln-plugin-nancheck]
nsteps = 50

[soln-bcs-wall] ; wall
type = no-slp-adia-wall

[soln-ics]
rho = rhoInf
u = Mach * ( 1.5*(1-y*y) + 0.2*exp(-y*y/0.05)*exp(-x*x/0.05)*exp(-z*z/0.05) ) ; 1.5*M*(1-(2/y/H)**2)
v = 0.2*exp(-y*y/0.05)*exp(-z*z/0.05)*exp(-x*x/0.05)
w = 0.2*exp(-y*y/0.05)*exp(-z*z/0.05)*exp(-x*x/0.05)
p = pInf

[soln-filter]
nsteps = 50 ; check time step
cutoff = 1 ; cut off order
order = 10 ; strength of the filter
alpha = 36.0 ; alpha = -log(ee) where ee is machine precision

```

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [3 September 2021 21:08 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/19 "2021-09-03T21:08:11Z")

</div>

Can you replace this `pass` by a print statement (contents do not matter):

> <https://github.com/PyFR/PyFR/blob/8bd1279f6f9833710c34577d787397385698b4de/pyfr/backends/openmp/base.py#L49>

and this except pass:

> <https://github.com/PyFR/PyFR/blob/8bd1279f6f9833710c34577d787397385698b4de/pyfr/backends/base/backend.py#L164>

by

```auto
except NotSuitableError as e:
    print(e)

```

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![Zhenyang](https://avatars.discourse-cdn.com/v4/letter/z/ec9cab/32.png) [@Zhenyang](https://pyfr.discourse.group/u/Zhenyang)\
**Post date:** [3 September 2021 21:23 UTC](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404/20 "2021-09-03T21:23:52Z")

</div>

Ah, I got the output message

```auto
Matrix too large for libxsmm
Matrix too dense for GiMMiK
Traceback (most recent call last):
......
    raise KeyError(f'Kernel "{name}" has no providers')
KeyError: 'Kernel "mul" has no providers'
......

```

Can that be fixed just increase the cpu number right?

Best wishes,  
Zhenyang

[Next page](https://pyfr.discourse.group/t/keyerror-on-3d-case-running-with-wsl-ubuntu-openmp/404.md?page=2)