# Installing PyFR with spack

**URL:** <https://pyfr.discourse.group/t/installing-pyfr-with-spack/546>\
**Category:** General\
**Created:** [8 February 2022 15:32 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546 "2022-02-08T15:32:03Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fab](https://avatars.discourse-cdn.com/v4/letter/f/51bf81/32.png) [@Fab](https://pyfr.discourse.group/u/Fab)\
**Post date:** [8 February 2022 15:32 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/1 "2022-02-08T15:32:03Z")

</div>

Hi @mlaufer,

> [@TGV Performance Numbers](https://pyfr.discourse.group/t/tgv-performance-numbers/407/17):
>
> Both installations used PyFR v1.12.2, GCC v10.2.0, libxsmm (at commit suggested in the performance guide), Open MPI v4.1.1, UCX v1.11.0. All packages were compiled from source using [Spack](https://github.com/spack/spack) and built to target the specific microarchitecture.

a bit late… did you install pyfr with spack as well? I did not see that this is available in the spack packages.

Best Regards  
Fab

---

<div class="post-metadata">

**Author:** ![mlaufer](https://yyz2.discourse-cdn.com/free1/user_avatar/pyfr.discourse.group/mlaufer/32/609_2.png) [@mlaufer](https://pyfr.discourse.group/u/mlaufer)\
**Post date:** [8 February 2022 15:58 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/2 "2022-02-08T15:58:25Z")

</div>

Hi,  
I created spack package recipes for PyFR, as well as the needed dependencies (GiMMiK), but i never pushed them to upstream Spack. @fdw and team, if you are interested in that I will revisit it and create a Spack PR.

The main sticking point I had was the special LIBXSMM commit that was required. Recently, I see that LIBXSMM has begun to release new versions once again (after taking a year off), so as long as the new version works out of the box, it should not be an issue. @fdw, how important is setting CODE\_BUF\_MAXSIZE=262144 when compiling LIBXSMM?

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [8 February 2022 16:18 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/3 "2022-02-08T16:18:55Z")

</div>

I am hoping that libxsmm 2.0 will be released in the next few months. This will include all of the relevant commits which PyFR needs (including ARM support). Although there have been a few released recently, these are all bugfixes to a legacy branch. The code buffer size is important for getting good performance at `p = 5` and beyond. Below that it does not make a difference.

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![mlaufer](https://yyz2.discourse-cdn.com/free1/user_avatar/pyfr.discourse.group/mlaufer/32/609_2.png) [@mlaufer](https://pyfr.discourse.group/u/mlaufer)\
**Post date:** [8 February 2022 16:48 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/4 "2022-02-08T16:48:18Z")

</div>

Thanks @fdw, in that case I will work on finalizing a PyFR Spack recipe, but will leave out LIBXSMM for now (as requiring a specific commit in a recipe is not a common practice in Spack). Once LIBXSMM is updated to v2.0, I will add a a variant with LibXSMM support. I will update here once the PR is pushed.

---

<div class="post-metadata">

**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\
**Post date:** [8 February 2022 17:58 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/5 "2022-02-08T17:58:21Z")

</div>

Sounds good. Just as an FYI in the next major release libxsmm will become a hard dependency for the OpenMP backend rather than an optional one.

Regards, Freddie.

---

<div class="post-metadata">

**Author:** ![Fab](https://avatars.discourse-cdn.com/v4/letter/f/51bf81/32.png) [@Fab](https://pyfr.discourse.group/u/Fab)\
**Post date:** [9 February 2022 06:53 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/6 "2022-02-09T06:53:48Z")

</div>

Hi, Thank you very much for your quick response!  
Regards Fab

---

<div class="post-metadata">

**Author:** ![mlaufer](https://yyz2.discourse-cdn.com/free1/user_avatar/pyfr.discourse.group/mlaufer/32/609_2.png) [@mlaufer](https://pyfr.discourse.group/u/mlaufer)\
**Post date:** [9 February 2022 13:34 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/7 "2022-02-09T13:34:07Z")

</div>

FYI  
I have pushed a PR for PyFR and GiMMiK packages:

> <https://github.com/spack/spack/pull/28847>
>
> Initial support for PyFR.
> @FreddieWitherden

As I mentioned above, LIBXSMM support is not yet implemented in the PyFR package, but will be added once LIBXSMM is updated to v2.0. I have tested the OpenMP backend, and CUDA backend succesfully, but I did not have the oppurtunity to test the HIP variant, I would appreciate if someone with access to relavant HW could give it a shot.

Optimized dependencies are built according to the [Performance Tuning guide](https://pyfr.readthedocs.io/en/latest/performance_tuning.html) (Parallel I/O support, optimized BLAS)

Assuming an optimized MPI implementation is already installed through Spack, a production ready install could be accomplished with a single command:

> spack install py-pyfr +cuda ^cuda@11.2.2 ^python+optimizations

@fdw, Let me know if you would like to assign one of the devs as a maintainer of these packages. If not, I will be happy to do it.

---

<div class="post-metadata">

**Author:** ![Fab](https://avatars.discourse-cdn.com/v4/letter/f/51bf81/32.png) [@Fab](https://pyfr.discourse.group/u/Fab)\
**Post date:** [9 February 2022 16:30 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/8 "2022-02-09T16:30:48Z")

</div>

Hi,  
I am not only new to pyfr, but also to spack 🙂 I understand from the [https://github.com/spack/spack/pull/28847](https://github.com/spack/spack/pull/28847)  
that a simple `git pull` will not provide this yet to my local repository. Is this right?  
Thank you!  
Fab

---

<div class="post-metadata">

**Author:** ![mlaufer](https://yyz2.discourse-cdn.com/free1/user_avatar/pyfr.discourse.group/mlaufer/32/609_2.png) [@mlaufer](https://pyfr.discourse.group/u/mlaufer)\
**Post date:** [9 February 2022 17:18 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/9 "2022-02-09T17:18:57Z")

</div>

Hi,  
Unless time-sensitive I would advise you to wait for the PR to be merged (a couple of days I expect). Otherwise you can use the Github CLI tool to test the PR. Luckily that is also available in spack:

```bash
spack install gh
spack load gh
gh pr checkout 28847
spack install py-pyfr

```

Good luck!

---

<div class="post-metadata">

**Author:** ![Fab](https://avatars.discourse-cdn.com/v4/letter/f/51bf81/32.png) [@Fab](https://pyfr.discourse.group/u/Fab)\
**Post date:** [10 February 2022 06:33 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/10 "2022-02-10T06:33:27Z")

</div>

Thank you very much! I will wait for the merge.

---

<div class="post-metadata">

**Author:** ![mlaufer](https://yyz2.discourse-cdn.com/free1/user_avatar/pyfr.discourse.group/mlaufer/32/609_2.png) [@mlaufer](https://pyfr.discourse.group/u/mlaufer)\
**Post date:** [15 February 2022 20:34 UTC](https://pyfr.discourse.group/t/installing-pyfr-with-spack/546/11 "2022-02-15T20:34:53Z")

</div>

FYI the Spack PR has been merged. I would appreciate feedback, especially for the HIP variant, which I was unable to test. Please submit issues to the Spack github and not here.  
Note that the Scotch mesh decomposition functionality is not yet usable ([see here](https://pyfr.discourse.group/t/pyfr-1-13-0-scotch-mesh-decomposition-error/553)).
