I was recently asked how Ampere GPUs compare to CPU, I have previously given some performance numbers, but decided to make more of a benchmark case for comparison. I ran a 40^3 hex mesh with p=3, which gives about 4.1 million solution points. The case I used was of course the TGV and below is the i…

From nvidia-smi the temperature was in the range of 230-240W (out of a max of 250W). But that doesn’t account for cooling etc, this is purely the device. The temperature of the GPU before I started was approx 30C and stabilised at 64/65C. So a significant amount of heat is having to be gotten rid of…

Sorry for the late response here. We had a memory configuration issue with our AMD Node that was killing performance. I ran tests on a dual socket AMD Epyc 7H12 (64c, 8 memory channels) node, which is the top of line Rome SKU. For reference this SKU is claimed to approach the performance of the top…

@mlaufer yep you are quite right on the M1 chip numbers. I previously made an error but clearly forgot to propagate it to that number. It should be ~21 M1s to one A100.

The AMD performance is held back by the lack of support for AVX-512. Specifically, GCC is unable (or unwilling) to vectorise the interface flux kernel on AVX2 CPUs. This represents a big hit, especially at single precision where the vectorisation is even more important. Additionally, libxsmm only…

2.302 /1.61=1.43 ，it seems close to 50%, why you say: [image] WillT: So the A100, as expected compares favourably with the V100, but given that the bandwidth is ~50% higher on the A100 I would’ve expected more. And I also run the case with A100 and MI100(one AMD’s GPU), the bandwidth of the…

Sure, I eyeballed it and it didn’t quite seem as high as 43% higher, but that still means we ~15% off from where we should be. So are those numbers single and double precision for the Mi100? I have tested PyFR Mi100 previously and don’t remember them being that far off. In other topics you were tal…

All tests were run up to t=1.0 with the NaN checker and Integration plugins disabled with double precision using pyfr=1.12.3 in env cuda:11.4.0-devel-ubuntu20.04 without any change. The execution time is got by "h5dump -d stats nse_tgv_3d_p3-1.00.pyfrs " just wall time. Another interesting thing is…

Can you use the data time at t=0 to get the compile time, you can then subtract this from the runtime from the t=1 solution file to get the true run time, just in case that is significant.

gimmik-cuda cublas gimmik-hip rocblas Setuptime 23.31 20.82 23.68 29.54 Walltime 341.16 405.25 503.62 612.61 Runtime 317.85 384.43 479.94 583.07 DoF 4.10E+06 4.10E+06 4.1…

@rishit2307 ran this test for me on an AMD Mi100 system and we got the following: single double GDoF/s 3.43 1.76 s/DoF(grid)/RHS/GPU 1.46\times10^{-9} 2.84\times10^{-9} The first row includes a factor of 5 in the DoF for the number of variables in Navier–Stokes, the second row doesn’t…

TGV Performance Numbers

General

Topic		Replies	Views
Poor TGV performance with GPU and MPI Cases hpc , cuda	5	276	1 June 2023
Example Datasets/configs for PyFR? Cases	14	920	11 March 2015
Bad Performance and warning General	9	511	19 May 2021
Memory on CUDA GPU backend General	14	1170	18 November 2021
Sd7003 case performance with OpenMP Cases incompressible , openmp	8	484	9 February 2020

TGV Performance Numbers

Related topics