I am interested in coupling Pyfr with other codes through a custom communication MPI based library. The general scheme to this would the following:
Replace internal MPI (mpi4py) communicators with one that I provide appropriately splitted.
During each solver time-step:
2.1) Have read access to the boundary/volume internal fields.
2.2) Have write access to the boundary/volume internal fields. That essentially means being able to modify source terms and boundary conditions programatically.
Is this something possible to do with Pyfr?
Maybe some interface like the one FEniCS provide so I can manually set up the case by writing a python script without using directly ‘pyrf’ command line tool?
I am interested in coupling Pyfr with other codes through a custom
communication MPI based library. The general scheme to this would the
following:
1) Replace internal MPI (mpi4py) communicators with one that I provide
appropriately splitted.
This should not be too difficult. Our MPI wrappers are in pyfr/mpi.py.
You should be able to augment these as appropriate. The only other
place where PyFR uses MPI communicators directly is in
pyfr/backends/base/packing.py
where there are a couple of references to COMM_WORLD.
2) During each solver time-step:
2.1) Have read access to the boundary/volume internal fields.
2.2) Have write access to the boundary/volume internal fields. That
essentially means being able to modify source terms and boundary
conditions programatically.
Accessing the volume after each time step is relatively simple. A
plugin can get you read access to the solution fields (with PyFR hiding
the backend specific copying code) or with a couple more lines you can
get read/write access. In addition PyFR also supports programmatic
source terms and so you can also consider augmenting/replacing this code
with something more elaborate.
In terms of boundaries you'll want to define your own boundary
condition. This is not too difficult and enables you to run your own
code between iterations and gives you full control over the 'kernel'
which is responsible for generating the ghost state.
Maybe some interface like the one FEniCS provide so I can manually set
up the case by writing a python script without using directly 'pyrf'
command line tool?
The pyfr command line tool is a relatively simple wrapper around the
PyFR API. Although we do not document it the interface is relatively
stable. Whilst some minor extensions may be required in order to
accomplish your goals, I do not think it is anything too difficult.
I realised I will have to be a bit invasive and change the baseline PyFR code and add files to the package source tree to achieve my goals (maybe I am wrong).
There are mainly 2 issues:
a ) Changing COMM_WORLD communicator manually. It would be would in later PyFR versions to not assume MPI_COMM_WORLD is the communicator used. This would be really easy to implement by allowing the pertinent functions to accept a ‘comm’ parameter that the user can provide.
b) Plugins directory is inside the package structure. Can I define my custom plugins in an arbitrary folder and register them in PyFR manually?
This is all for the time being. Thanks in advance.
I realised I will have to be a bit invasive and change the baseline PyFR
code and add files to the package source tree to achieve my goals (maybe
I am wrong).
There are mainly 2 issues:
a ) Changing COMM_WORLD communicator manually. It would be would in
later PyFR versions to not assume MPI_COMM_WORLD is the communicator
used. This would be really easy to implement by allowing the pertinent
functions to accept a 'comm' parameter that the user can provide.
We can certainly do this in a future release; heretofore we have had no
uses cases which require changing the communicator and we generally try
to avoid adding functionality before we have a good use case.
b) Plugins directory is inside the package structure. Can I define my
custom plugins in an arbitrary folder and register them in PyFR manually?
Yes. The only requirement is that the plugins be a subclass of
BasePlugin and known to Python at the time when the integrator class is
being constructed. Practically, this means that you simply need to have
imported the class somewhere in your code (where does not matter).
Let us know if you need us to extend the plugin functionality at all.