Error at startup for opencl backend

General
1

All,

I get the following traceback error when attempting to run with the attached *.ini file:

Traceback (most recent call last):
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 257, in <module>
main()
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 110, in main
args.process(args)
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 235, in process_run
args, NativeReader(args.mesh), None, Inifile.load(args.cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 216, in _process_common
solver = get_solver(backend, rallocs, mesh, soln, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/__init__.py", line 16, in get_solver
return get_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/__init__.py", line 43, in get_integrator
return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/controllers.py", line 14, in __init__
super().__init__(*args, **kwargs)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/steppers.py", line 8, in __init__
super().__init__(*args, **kwargs)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/base.py", line 12, in __init__
super().__init__(*args, **kwargs)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/base.py", line 59, in __init__
self.system = systemcls(backend, rallocs, mesh, initsoln, nreg, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", line 58, in __init__
int_inters = self._load_int_inters(rallocs, mesh, elemap)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", line 116, in _load_int_inters
self.cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/navstokes/inters.py", line 13, in __init__
super().__init__(be, lhs, rhs, elemap, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvecdiff/inters.py", line 12, in __init__
super().__init__(be, lhs, rhs, elemap, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvec/inters.py", line 11, in __init__
super().__init__(be, lhs, elemap, cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/inters.py", line 37, in __init__
for etype, eidx, fidx, flags in lhs)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/inters.py", line 37, in <genexpr>
for etype, eidx, fidx, flags in lhs)
KeyError: ‘’

Is there a typo somewhere in my *.ini file that I’m overlooking? The mesh is 2-D consisting of entirely quads. Any assistance would be appreciated.

Best Regards,

Pointwise, Inc. |
Zach Davis
Pointwise®, Inc.
Sr. Engineer, Sales & Marketing
213 South Jennings Avenue
Fort Worth, TX 76104-1107

E: zach.davis@pointwise.com
P: (817) 377-2807 x1202
F: (817) 377-2799
|

  • | - |

enc

vortex-street.ini (2.79 KB)

2

Hi Zach,

I suspect that the problem is with the mesh; specifically in the interfaces list there is an interface listed whose type is empty (where is should be "quad").

Regards, Freddie.

3

Hi Freddie,

It seems our pyfr plug-in may need to be updated then. I was able to get something started using the Gmsh plugin, and then importing the mesh into PyFR’s format. I’ll have a developer look into it.

On another unrelated note, how do you determine the appropriate local work size for one dimensional and two dimensional pointwise kernels for a given graphics card using the OpenCL backend (i.e. where does one find appropriate values for local-size-1d and local-size-2d)?

Best Regards,

4

Hi Zach,

Hi Freddie,

It seems our pyfr plug-in may need to be updated then. I was able to
get something started using the Gmsh plugin, and then importing the mesh
into PyFR’s format. I’ll have a developer look into it.

Let us know if you need any help. As I said the problem looks to be in the "con_p<x>" data set inside the HDF5 file.

On another unrelated note, how do you determine the appropriate local
work size for one dimensional and two dimensional pointwise kernels for
a given graphics card using the OpenCL backend (i.e. where does one find
appropriate values for local-size-1d and local-size-2d)?

Usually the defaults work well, very well in fact. Those options were added to handle some pathological cases I was encountering where the elements in the mesh have a poor numbering that was resulting in a large number of cache misses. Here it can help to reduce the work size as it reduces pressure on the caches.

Chances are you'll never need it.

Regards, Freddie.

5

Alright,

Well, I’m now getting the following traceback which leads me to believe that perhaps these work sizes are not appropriate for my card:

Wrote file /Users/zdavis/Desktop/vortex-street/vortex-street-0.00.pyfrs at time 0.0 for mesh /Users/zdavis/Desktop/vortex-street/vortex-street.pyfrm.
Traceback (most recent call last):
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 257, in <module>
main()
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 110, in main
args.process(args)
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 235, in process_run
args, NativeReader(args.mesh), None, Inifile.load(args.cfg)
File "/Users/zdavis/Applications/PyFR/pyfr/pyfr", line 227, in _process_common
solver.run()
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/base.py", line 199, in run
self.advance_to(t)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/controllers.py", line 70, in advance_to
idxcurr = self.step(self.tcurr, dt)
File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/steppers.py", line 79, in step
rhs(t, r0, r2)
File "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvecdiff/system.py", line 33, in rhs
runall([q1, q2])
File "/Users/zdavis/Applications/PyFR/pyfr/backends/base/backend.py", line 176, in runall
self.queue_cls.runall(sequence)
File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/types.py", line 114, in runall
q._exec_nowait()
File "/Users/zdavis/Applications/PyFR/pyfr/backends/base/types.py", line 326, in _exec_nowait
self._exec_item(*self._items.popleft())
File "/Users/zdavis/Applications/PyFR/pyfr/backends/base/types.py", line 311, in _exec_item
item.run(self, *args, **kwargs)
File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/provider.py", line 50, in run
fun(queue.cl_queue_comp, gs, ls, *narglst)
File "/Users/zdavis/Applications/PyFR/lib/python3.6/site-packages/pyopencl/cffi_cl.py", line 1984, in __call__
return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
File "<generated function enqueue_knl_intconu>", line 107, in enqueue_knl_intconu
File "/Users/zdavis/Applications/PyFR/lib/python3.6/site-packages/pyopencl/cffi_cl.py", line 2168, in enqueue_nd_range_kernel
global_work_size, local_work_size, c_wait_for, num_wait_for))
File "/Users/zdavis/Applications/PyFR/lib/python3.6/site-packages/pyopencl/cffi_cl.py", line 632, in _handle_error
raise e
pyopencl.cffi_cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_WORK_GROUP_SIZE

Is this the culprit, or have I done something else wrong?

Best Regards,

6

Hi Zach,

It could be that your OpenCL implementation imposes a minimum size. Do the defaults work? (Just remove the blocks from the file and PyFR will fill out the defaults for you.)

Regards, Freddie.

7

Hi Freddie,

I’ve commented out both parameters in the *.ini file, but I still get the same error… The OpenMP backend seems to work.

Best regards,

8

Hi Zach,

You can try setting the 1D block size to 128 which will override the default of 64 (which is already quite large).

Regards, Freddie.

9

Hi Zach,

So a tool like opencl-info:

should be able to tell you what your hardware supports. However, these issues are unusual and not something we have observed with other platforms.

Regards, Freddie.

10

Hi Zach,

Sorry for the late response. I have been debugging this issue over the
past week and believe it to be fixed in PyFR 1.6.0 which was released
last week. Please give it a go and let me know if it works.

Regards, Freddie.