PyFR GPU Performance

General
1

Hello,

I am attempting to assess the performance of my PyFR setup through comparison to Redirecting , namely Table 3:

image
image761×237 32.5 KB

Here are some details about my test case:

  • PyFR v3.0
  • GH200 GPU
  • GPU Count (same as N_ranks): 2
  • Cuda-aware MPI
  • Double precision
  • No plugins besides soln writer every 1E5 steps
  • 3D Comp. NS solver
  • 2.72E5 tet element mesh
  • Order = 8
  • Wall-time: 9.68E4 s
  • RHS evals: 4E5

To compute DoF/s, I use N_{dof}N_{rhseval}/T_{wall} .
If, I define N_{dof} = N_{ele} N_{pts/ele} N_{var}, then my performance is 4.6 GDoF/s/GPU compared to the paper’s reported ~6 GDoF/s/GPU. I’ve also verified that my local workload (soln pts per GPU) matches that of the paper’s performance.

If N_{dof} does not include N_{var}, then I am also a factor of 5 slower.

The language of the paper seems to suggest that N_{var} isn’t considered in the DoF calculation. Could you clarify if it is or isn’t?

Additionally, do you have suggestions on what to look into for explaining/removing the difference in performance between my case and the published results?

Thank you,

Jay

2

The paper used order = 7 as per

The computational mesh consisted of tetrahedral elements, with seventh-order solution polynomials used to represent the solution within each element"

Regards, Freddie.

3

Thank you for the response.
I apologize if I’m not understanding completely.

Could you clarify if the GDoF/s calculation includes N_{var} = 5 ?

Thank you,

Jay

4

Yes, it includes the factor of 5.

Regards, Freddie.