PyFR simulating NACA0021 airfoil in deep stall

Hello everyone,

I am trying to repeat the NACA0021 test of Witherden, Jameson (2020) (https://arc.aiaa.org/doi/abs/10.2514/1.J058434), Sec D. Basically, starting with order = 1 without antialiasing with tend = 100.0 and with Re = 27000 (i.e. mu = 0.000037), I attempted restarting from there with order = 4 activating antialiasing for another 300 t (i.e. tend = 400) and modifying the Re to 270000 (mu = 0.0000037).

The first part of the simulation was completed without errors. However, the restarted simulation crashed after time 107.28, with the following error:

RuntimeError: Minimum sized time step rejected

From the forum, I already noticed that such kind of error can happen when restarting the simulation with a different order (see page https://pyfr.discourse.group/t/naca-0021-airfoil-in-deep-stall-minimum-sized-time-step-rejected/295).

I adopted the p4_aa.ini config file as a template for my simulation, available within the supplemental material of Park, Witherden, and Vincent (2017) (see https://arc.aiaa.org/doi/full/10.2514/1.J055304). The relevant differences between the warmup and restart phase are as follows:

warmup

[constants]
...
mu = 0.000037
...
[solver]
...
order = 1
#anti-alias = flux, surf-flux
...
[solver-time-integrator]
...
tend = 100.0
...

restart

[constants]
...
mu = 0.0000037
...
[solver]
...
order = 4
anti-alias = flux, surf-flux
...
[solver-time-integrator]
...
tend = 400.0
...

I kept the rest of the file unchanged, therefore the filtering section for both config files is:

[soln-filter]
freq = 0
cutoff = 0
order = 16
alpha = 36

Can someone give me some hints on how to heal/monitor the instability that is making the code crashing?

For sake of clarity, I adopted the mesh naca0021_p4_4c.msh for this test, that is available within the supplemental material of the paper Park, Witherden, and Vincent (2017).

Regards,
Federico Cipolletta.

So I’ve just ran the 7c case on our cluster with your .ini files and everything looks good. On 12 V100 GPUs the first part at p = 1 runs as:

100.0% [===========================>] 100.00/100.00 ela: 01:44:33 rem: 00:00:00

and the restart with p = 4 is currently running as:

27.6% [++++++=>                   ] 110.60/400.00 ela: 12:38:00 rem: 344:59:46

which is past the point where you are observing issues with a span of 4c. I will let this run for a few more hours and then take a look at the span 4c case.

Regards, Freddie.

1 Like

I have now ran the 4c case with 12 V100’s GPUs. The start-up went as:

100.0% [===========================>] 100.00/100.00 ela: 01:26:54 rem: 00:00:00

which is reasonable (although p = 1 is somewhat pathological and so the speed up is less than the 7/4 we might expect). Restarting at p = 4 I observed:

31.7% [++++++==>                  ] 126.96/400.00 ela: 19:28:46 rem: 197:17:17

which has made it nicely past the point where you were observing issues. The 7c case is also still running without issues with:

32.7% [++++++==>                  ] 130.91/400.00 ela: 36:44:03 rem: 319:45:38

In terms of next steps can you try moving to the latest (Git master) version of PyFR, reimporting your mesh, and re-running?

Regards, Freddie.

Hello Freddie,

and thanks for your kind answer. I am actually using PyFR v 1.12.0. Unfortunately, our new cluster does not have internet access, therefore I cannot use git. However, I noticed that the most recent version of PyFR is 1.12.1, so I can try to do that manually, substituing this latter one in my Python environment via

python setup.py build
python setup.py install

from the newest version.

I’ll try that ASAP and will soon come back to you.

Best,
Federico Cipolletta.

Hi Federico,

Yes, its definitely worth giving v1.12.1 a try. I think there was an issue with the mesh importer in v1.12.0 that led to erroneous linearisation of curved element meshes that could be causing your stability problems.

Peter

Hello everyone,

I tried installing PyFR v 1.12.1 on top of my python environment which was containing the PyFR v 1.12.0. However, the upgrade done via simply

python setup.py build
python setup.py install

seems not to be working out-of-the-box. From the second command, I am obtaining:

...
running install_data
copying pyfr/__main__.py -> build/bdist.linux-x86_64/egg/
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pyfr.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
  File "build/bdist.linux-x86_64/egg/pyfr/solvers/base/system.py", line 80
    if (m := re.match(f'spt_(.+?)_p{rallocs.prank}$', f)):
          ^
SyntaxError: invalid syntax

  File "build/bdist.linux-x86_64/egg/pyfr/writers/native.py", line 98
    if (m := re.match(bn, f)):
          ^
SyntaxError: invalid syntax

  File "build/bdist.linux-x86_64/egg/pyfr/integrators/base.py", line 71
    if (m := re.match('soln-plugin-(.+?)(?:-(.+))?$', s)):
          ^
SyntaxError: invalid syntax

  File "build/bdist.linux-x86_64/egg/pyfr/integrators/dual/pseudo/multip.py", line 79
    if (m := re.match(f'solver-(.*)-mg-p{l}$', s)):
          ^
SyntaxError: invalid syntax

  File "build/bdist.linux-x86_64/egg/pyfr/rank_allocator.py", line 58
    if (m := re.match(r'con_p(\d+)p(\d+)$', f)):
          ^
SyntaxError: invalid syntax

  File "build/bdist.linux-x86_64/egg/pyfr/partitioners/base.py", line 64
    if (mi := re.match(r'con_p(\d+)$', f)):
           ^
SyntaxError: invalid syntax

zip_safe flag not set; analyzing archive contents...
creating dist
...

and, even if the pip freeze reports the correct new version of PyFR (i.e. 1.12.1), when I attempt to run the NACA0021 test, I obtain the following error:

...
  File "/davinci-1/home/cipollettaf/pyfr_mlnx/lib/python3.7/site-packages/pyfr-1.12.1-py3.7.egg/pyfr/partitioners/base.py", line 64
    if (mi := re.match(r'con_p(\d+)$', f)):
           ^
SyntaxError: invalid syntax
...

repetead as many time as per the number of GPUs I am running on. This error, seems to happen at mesh importt, because the file .pyfrm is not produced.

Are you aware of differences between version 1.12.0 and 1.12.1 that can produce those errors?

If I try to install PyFR 1.12.1 via pip install, I get an error, where it seems that my Python (3.7.9) is not enough for the code that is requiring Python >= 3.8.

I verified that the setup.py script is actually explicitly requiring

python_requires='>=3.8'

on line 157. However, simply modifiying this line does not heal the issues and the errors remain. It seems that the changes in the mesh importer (basically using := in some if conditions) implicitly introduced the requirement of Python 3.8 (see https://towardsdatascience.com/when-and-why-to-use-over-in-python-b91168875453). I think that you may want to update the requirements in the official documentation, if you are not trying to avoid the Python 3.8 syntax.

Best,
Federico Cipolletta.

This is the fix for the issue I mentioned: Bug fix. · PyFR/PyFR@8f59954 · GitHub

Yes, we require Python 3.8 (although I believe this change came in with 1.12.0, so if you were using this version you should not be encountering these problems) and this carries through to PyPI which notes the dependency on 3.8 or later:

I am unsure why the setup.py build/install route does not catch this although generally pip install . is a better bet than calling setup.py manually.

In terms of next steps I would verify what release you’ve been running heretofore, as it may not have been 1.12.0. Then, bump your Python version to 3.8 or later and proceed with installing 1.12.1.

Regards, Freddie.

Unfortunately for me, as we have restrictions regarding the internet connection from within the company, adopting pip is not straightforward.

However, installing from scratch a new Python environment and proceeding with installation of the required packages from source, worked for me. The total list of packages that I have installed on my new environment is a little longer than what is reported in the requirements on documentation and is the following:

appdirs==1.4.4
Cython==3.0a6       (Required by numpy)
gimmik==2.1
h5py==3.3.0
Mako==1.1.4
MarkupSafe==1.1.1   (Required by Mako)
mpi4py==3.0.3
numpy==1.20.3
pyfr==1.12.1
pytools==2021.2.6

I am now running the p1 warmup of the NACA0021 test with the new environemnt and will then attempt switching again to p4.

I will come back to you as soon as I have results.

Best,
Federico Cipolletta.

Hello everyone,

It seems that adopting the latest version of PyFR (v 1.12.1) made my simulation stable.

To recap, I am running the 4c mesh case, with a warmup with order 1 up to time t=100 and restarting from there switching to order 4, and activating anti-aliasing. I am running the simulation on 12 NVIDIA A100 GPUs.

The warmup was completed with the following stdout:

 100.0% [===========================>] 100.00/100.00 ela: 01:04:45 rem: 00:00:00

The differences of the config files (warmup vs restart) that I am using are as follows:

--- P1_warmup/p1_warmup.ini     2021-07-20 09:41:39.000000000 +0200
+++ P4_4c/p4_aa.ini     2021-07-21 12:42:34.000000000 +0200
@@ -12,8 +12,8 @@
 [constants]
...
-mu = 0.000037
+mu = 0.0000037
@@ -22,9 +22,9 @@
 [solver]
...
-order = 1
+order = 4
+anti-alias = flux, surf-flux
...
 [solver-time-integrator]
...
@@ -32,7 +32,8 @@
-tend = 100.0
+tend = 400.0
...
[soln-plugin-writer]
...
@@ -77,7 +78,7 @@
-basename = naca_p1_4c-{t:.2f}
+basename = naca_p4_4c-{t:.2f}
...

The actual status of the restart is as follows:

  34.9% [++++++===>                 ] 139.70/400.00 ela: 20:25:31 rem: 133:56:13

NOTE: if one forgets to activate the anti-aliasing, the simulation does crash and this is probably due to the fact that the turbulence created but the deep stall position of the NACA airfoil profile with respect to the velocity of the fluid needs some artificial dissipation.

Thanks for the help!

Glad to hear you got it working. Just a note regarding:

“if one forgets to activate the anti-aliasing, the simulation does crash and this is probably due to the fact that the turbulence created but the deep stall position of the NACA airfoil profile with respect to the velocity of the fluid needs some artificial dissipation.”

Anti-aliasing doesn’t add artificial dissipation, rather it suppresses spurious transfer of energy from un-resolved modes into the highest-energy resolved modes (which can then lead to instability).