I also use openmp/libxsmm run the TGV case , there is a big efficiency loss of multi-thread parallelism. Here is my machine
And this is expected time
Is this OK?
And this is expected time
Is this OK?
Have a look at some of the discussion in the TGV performance topic as well as other topics. This is fairly common and there are some nuanced reasons for it that are dependent on architecture.
You are running with too many threads for such a small case (or have other OpenMP applications sharing the same cores). For reference with libxsmm on a recent Intel chip:
❯ OMP_NUM_THREADS=2 pyfr run -bopenmp -p euler_vortex_2d.pyfrm euler_vortex_2d.ini 100.0% [=============================================================================>] 100.00/100.00 ela: 00:00:06 rem: 00:00:00
Regards, Freddie.
Although your CPU claims to be a 32-core chip it is actually four eight core chips, and should be treated as such. Hence you will want to run with four MPI ranks with eight threads per rank with everything correctly pinned to the appropriate NUMA zone.
The specifics of how to accomplish this depend on your MPI and OpenMP runtimes. Although I would strongly recommend figuring it out, if you are in a rush you can obtain reasonable performance by running with 32 MPI ranks and one thread per rank.
Regards, Freddie.
1 Like
Thank you very much ! !
Excuse me, I run the pyfr with mpi, using command like “mpiexec -np 16 pyfr …”
It shows this in terminal ,while the process is on ,what’s the problem ?Could you help me ?
If I set gimmik=1 in cuda benckend , -b cuda , will I use cublas to run pyfr?
Thank you ! I notice that you get a performance number GDoF/s, How can I transfer it to "Gflop/s” ?
I don’t think there is an easy conversion from GDoF/s to GFlop/s as you would have to work out how many flops are being performed. This is not trivial to do and is the main reason behind using GDoF/s instead.
If not (from the source code it seems not), is there an easy way i can use cublas to run pyfr?
On the cuda backend gimmik-max-nnz is not used. If you want to force pyfr to use cublas you can raise a NotSuitableError here: PyFR/gimmik.py at 59196ff0785e27adafb387a7a78d7bab80b2b989 · PyFR/PyFR · GitHub
These errors are coming from your MPI library. Try recompiling that and mpi4py.
Regards, Freddie.
A post was split to a new topic: Question on the functionlaity of GiMMiK
So as you said ,pyfrs and pyfrm file are in the HDF5 format, can .pyfrs file be open with paraview directly?
No, pyfr files are not currently supported by Paraview. Saying something uses an HDF5 format is like saying it is an xml file, xml is the format the data is written with but doesn’t tell you anything about how the data in the file is structured. pyfrm and pyfrs files are written using HDF5 and have very specific structures.
Something you can do is open them in Matlab, .mat files also use the HDF5 format.
Thank you very much!!!