# Adapting to Changes in PyFR 1.4.0 Release **URL:** \ **Category:** General\ **Created:** [5 May 2016 23:00 UTC](https://pyfr.discourse.group/t/adapting-to-changes-in-pyfr-1-4-0-release/110 "2016-05-05T23:00:53Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Zach\_Davis1](https://avatars.discourse-cdn.com/v4/letter/z/3ec8ea/32.png) [@Zach\_Davis1](https://pyfr.discourse.group/u/Zach_Davis1)\ **Post date:** [5 May 2016 23:00 UTC](https://pyfr.discourse.group/t/adapting-to-changes-in-pyfr-1-4-0-release/110/1 "2016-05-05T23:00:53Z")
All, I was attempting to run a case using PyFR—it’s been awhile and a few things have change—but it appears that PyFR doesn’t like my clBLAS library any more under OS X. The pyopencl module seems to have compiled fine when I installed it, and the path to the libclBLAS.dylib file is in my DYLD\_LIBRARY\_PATH environment. Unfortunately, PyFR seems to be having some issue locating it. Is there another environment variable, or parameter that can be set somewhere to explicitly let PyFR know where it can find this library in my environment? I’ve included the stack trace below: `(venv) [zdavis@Minerva naca0012]$ pyfr run -b opencl -p naca_0012_2d.pyfrm naca_0012_2d.ini` ```auto Traceback (most recent call last): File "/Users/zdavis/Applications/PyFR/pyfr/ctypesutil.py", line 22, in load_library return ctypes.CDLL(lname) File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ctypes/ __init__.py", line 347, in __init__ self._handle = _dlopen(self._name, mode) OSError: dlopen(libclBLAS.dylib, 6): image not found During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 252, in main() File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 109, in main args.process(args) File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 230, in process_run args, NativeReader(args.mesh), None, Inifile.load(args.cfg) File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 208, in _process_common backend = get_backend(args.backend, cfg) File "/Users/zdavis/Applications/PyFR/pyfr/backends/ __init__.py", line 12, in get_backend return subclass_where(BaseBackend, name=name.lower())(cfg) File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/base.py", line 79, in __init__ self._providers = [k(self) for k in kprovs] File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/base.py", line 79, in self._providers = [k(self) for k in kprovs] File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/clblas.py", line 65, in __init__ self._wrappers = ClBLASWrappers() File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/clblas.py", line 14, in __init__ lib = load_library('clBLAS') File "/Users/zdavis/Applications/PyFR/pyfr/ctypesutil.py", line 31, in load_library raise OSError('Unable to load {0}'.format(name)) OSError: Unable to load clBLAS ``` I’m also getting an invalid character when running using the openmp backend. Again the stack trace is provided below: `(venv) [zdavis@Minerva naca0012]$ mpirun -np 4 pyfr run -b openmp -p naca_0012_2d.pyfrm naca_0012_2d.ini` ```auto Traceback (most recent call last): File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 252, in main() File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 109, in main args.process(args) File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 230, in process_run args, NativeReader(args.mesh), None, Inifile.load(args.cfg) File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 214, in _process_common solver = get_solver(backend, rallocs, mesh, soln, cfg) File "/Users/zdavis/Applications/PyFR/pyfr/solvers/ __init__.py", line 14, in get_solver return get_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg) File "/Users/zdavis/Applications/PyFR/pyfr/integrators/ __init__.py", line 43, in get_integrator return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg) File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/controllers.py", line 12, in __init__ super(). __init__ (*args, **kwargs) File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/steppers.py", line 8, in __init__ super(). __init__ (*args, **kwargs) File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/base.py", line 12, in __init__ super(). __init__ (*args, **kwargs) File "/Users/zdavis/Applications/PyFR/pyfr/integrators/base.py", line 57, in __init__ self.system = systemcls(backend, rallocs, mesh, initsoln, nreg, cfg) File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", line 54, in __init__ bc_inters = self._load_bc_inters(rallocs, mesh, elemap) File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", line 151, in _load_bc_inters self.cfg) File "/Users/zdavis/Applications/PyFR/pyfr/solvers/navstokes/inters.py", line 226, in __init__ self._tpl_c['cpTt'], = self._eval_opts(['cpTt']) File "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvec/inters.py", line 98, in _eval_opts return [npeval(cfg.getexpr(sect, k), cc) for k in opts] File "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvec/inters.py", line 98, in return [npeval(cfg.getexpr(sect, k), cc) for k in opts] File "/Users/zdavis/Applications/PyFR/pyfr/inifile.py", line 79, in getexpr raise ValueError('Invalid characters in expression') ValueError: Invalid characters in expression ``` Is this an issue with my \*.ini file? I’ve attached it for reference… Thanks! Best Regards, Zach Davis enc [naca\_0012\_2d.ini](https://pyfr.discourse.group/uploads/short-url/2a2FJyVQHpFN7mbdJBvjBmqheTY.ini) (2.53 KB) ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [6 May 2016 04:43 UTC](https://pyfr.discourse.group/t/adapting-to-changes-in-pyfr-1-4-0-release/110/2 "2016-05-06T04:43:53Z")
Hi Zach, > I was attempting to run a case using PyFR—it’s been awhile and a few > things have change—but it appears that PyFR doesn’t like my clBLAS > library any more under OS X. The pyopencl module seems to have compiled > fine when I installed it, and the path to the libclBLAS.dylib file is in > my DYLD\_LIBRARY\_PATH environment. Unfortunately, PyFR seems to be > having some issue locating it. Is there another environment variable, > or parameter that can be set somewhere to explicitly let PyFR know where > it can find this library in my environment? I’ve included the stack > trace below: > > (venv) [zdavis@Minerva naca0012]$ pyfr run -b opencl -p > naca\_0012\_2d.pyfrm naca\_0012\_2d.ini > Traceback (most recent call last): >   File "/Users/zdavis/Applications/PyFR/pyfr/ctypesutil.py", line 22, in > load\_library >     return ctypes.CDLL(lname) >   File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ctypes/\_\_init\_\_.py", > line 347, in \_\_init\_\_ >     self.\_handle = \_dlopen(self.\_name, mode) > OSError: dlopen(libclBLAS.dylib, 6): image not found > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 252, in > \ >     main() >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 109, in > main >     args.process(args) >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 230, in > process\_run >     args, NativeReader(args.mesh), None, Inifile.load(args.cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 208, in > \_process\_common >     backend = get\_backend(args.backend, cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/backends/\_\_init\_\_.py", line > 12, in get\_backend >     return subclass\_where(BaseBackend, name=name.lower())(cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/base.py", > line 79, in \_\_init\_\_ >     self.\_providers = [k(self) for k in kprovs] >   File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/base.py", > line 79, in \ >     self.\_providers = [k(self) for k in kprovs] >   File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/clblas.py", > line 65, in \_\_init\_\_ >     self.\_wrappers = ClBLASWrappers() >   File "/Users/zdavis/Applications/PyFR/pyfr/backends/opencl/clblas.py", > line 14, in \_\_init\_\_ >     lib = load\_library('clBLAS') >   File "/Users/zdavis/Applications/PyFR/pyfr/ctypesutil.py", line 31, in > load\_library >     raise OSError('Unable to load {0}'.format(name)) > OSError: Unable to load clBLAS The environmental variable PYFR\_LIBRARY\_PATH can be used to specify additional search paths for dynamic libraries. Hence, export PYFR\_LIBRARY\_PATH=/path/to/dir/with/clBLAS/ should do the trick assuming that there is a library (or a suitable symlink) to libclBLAS.dylib there. Although this is not 'recommended' in the sense that it is better to install the library to a location that is on the search path of the dynamic loader it is 100% supported. > I’m also getting an invalid character when running using the openmp > backend. Again the stack trace is provided below: > > (venv) [zdavis@Minerva naca0012]$ mpirun -np 4 pyfr run -b openmp -p > naca\_0012\_2d.pyfrm naca\_0012\_2d.ini > Traceback (most recent call last): >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 252, in > \ >     main() >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 109, in > main >     args.process(args) >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 230, in > process\_run >     args, NativeReader(args.mesh), None, Inifile.load(args.cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/scripts/pyfr", line 214, in > \_process\_common >     solver = get\_solver(backend, rallocs, mesh, soln, cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/solvers/\_\_init\_\_.py", line > 14, in get\_solver >     return get\_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/integrators/\_\_init\_\_.py", > line 43, in get\_integrator >     return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg) >   File > "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/controllers.py", > line 12, in \_\_init\_\_ >     super().\_\_init\_\_(\*args, \*\*kwargs) >   File > "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/steppers.py", line > 8, in \_\_init\_\_ >     super().\_\_init\_\_(\*args, \*\*kwargs) >   File "/Users/zdavis/Applications/PyFR/pyfr/integrators/std/base.py", > line 12, in \_\_init\_\_ >     super().\_\_init\_\_(\*args, \*\*kwargs) >   File "/Users/zdavis/Applications/PyFR/pyfr/integrators/base.py", line > 57, in \_\_init\_\_ >     self.system = systemcls(backend, rallocs, mesh, initsoln, nreg, cfg) >   File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", > line 54, in \_\_init\_\_ >     bc\_inters = self.\_load\_bc\_inters(rallocs, mesh, elemap) >   File "/Users/zdavis/Applications/PyFR/pyfr/solvers/base/system.py", > line 151, in \_load\_bc\_inters >     self.cfg) >   File > "/Users/zdavis/Applications/PyFR/pyfr/solvers/navstokes/inters.py", line > 226, in \_\_init\_\_ >     self.\_tpl\_c['cpTt'], = self.\_eval\_opts(['cpTt']) >   File > "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvec/inters.py", line > 98, in \_eval\_opts >     return [npeval(cfg.getexpr(sect, k), cc) for k in opts] >   File > "/Users/zdavis/Applications/PyFR/pyfr/solvers/baseadvec/inters.py", line > 98, in \ >     return [npeval(cfg.getexpr(sect, k), cc) for k in opts] >   File "/Users/zdavis/Applications/PyFR/pyfr/inifile.py", line 79, in > getexpr >     raise ValueError('Invalid characters in expression') > ValueError: Invalid characters in expression > > Is this an issue with my \*.ini file? I’ve attached it for reference… > Thanks! PyFR now allows a variety of expressions, such as those used to define boundary conditions, to be functions of both time and space. A consequence of this is that our expression language has to be the lowest common denominator of what is supported across Python/OpenCL/CUDA/C. This means that PyFR is no longer able to accept '^' for exponentiation. Instead, one must use the pow(x, y) function to specify exponents. For Sutherland's law it looks as if a couple of the constants are missing. You need to define: cpTref and cpTs in the [constants]. The only other thing is that we have modernised the syntax used in [solver-plugin-writer] basename. Specifically, the string is now passed to the format() method hence: basename = naca\_0012\_2d-(t:.2f) should be basename = naca\_0012\_2d-{t:.2f} Hope that helps. Regards, Freddie. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [6 May 2016 17:24 UTC](https://pyfr.discourse.group/t/adapting-to-changes-in-pyfr-1-4-0-release/110/3 "2016-05-06T17:24:48Z")
Hi Zach, My understanding is that the Intel Iris Pro GPUs do not support double precision. This is something we should probably check for in the OpenCL backend (there is an API call to determine if double precision compute is supported or not). Regards, Freddie. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [6 May 2016 17:51 UTC](https://pyfr.discourse.group/t/adapting-to-changes-in-pyfr-1-4-0-release/110/4 "2016-05-06T17:51:12Z")
Hi Zach, > Thanks Freddie, I had device-id set to local-rank, and I intended a > value of 1 here. All is well. It appears that throwing the entire > grid as a single partition on my discrete GPU via OpenCL runs much > slower than partitioning the grid into 4 parts and running on the > CPU. Does this make sense to you? Any idea as to why I’m not > getting solution output files? In general 2D cases are not great from a benchmark standpoint. They tend to be highly limited by available memory bandwidth and are often too small to sufficiently load a capable GPU. Moreover, on a good day it is possible to fit an extremely large portion of the working set into the last level cache of a modern CPU; as this point it really is no contest. Hence, it is not unusual for modest CPUs to substantially outperform high end GPUs in 2D cases. For file output: rename the section to [soln-plugin-writer] and make sure that the basename uses {} for variable substitution. Regards, Freddie.