# What's the relationship between Gimmik and cublas? **URL:** \ **Category:** General\ **Created:** [9 February 2022 06:07 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550 "2022-02-09T06:07:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![luli](https://avatars.discourse-cdn.com/v4/letter/l/5f9b8f/32.png) [@luli](https://pyfr.discourse.group/u/luli)\ **Post date:** [9 February 2022 06:07 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/1 "2022-02-09T06:07:50Z")
I read the paper [Redirecting](https://doi.org/10.1016/j.cpc.2015.12.012) and, it seems that GIMMIk is better than cublas? Does the new version of pyfr still need to use the library cublas or not? ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [9 February 2022 08:56 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/2 "2022-02-09T08:56:02Z")
Gimmik is great in some circumstances, and with some of our ongoing work the set of circumstances that is works well for is always increasing. In the gimmik [mul definition in pyfr](https://github.com/PyFR/PyFR/blob/c10e0e64e8ca6badfb39b208cb5a9f42bc88c73d/pyfr/backends/cuda/gimmik.py#L15) there are some checks that are made, if the matrix doesn’t pass those checks a `NotSuitableError` is thrown. This is picked up by PyFR [here](https://github.com/PyFR/PyFR/blob/c10e0e64e8ca6badfb39b208cb5a9f42bc88c73d/pyfr/backends/base/backend.py#L154), and then the next provider is tried. What this means is PyFR will normally first try to use GiMMiK and then try cuBLAS as a fallback. @fdw may have a view on this, but I don’t see a situation where we completely remove cuBLAS as I think it is unlikely that GiMMiK will be as good as cuBLAS for large truly dense matrices, i.e those that occur for tets with order \> ~5. ---
**Author:** ![luli](https://avatars.discourse-cdn.com/v4/letter/l/5f9b8f/32.png) [@luli](https://pyfr.discourse.group/u/luli)\ **Post date:** [11 February 2022 03:25 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/3 "2022-02-11T03:25:12Z")
Thank you! How can I choose to use cuBLAS only? I want to compare the GiMMiK and cuBLAS performance with my specific case. ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [11 February 2022 07:08 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/4 "2022-02-11T07:08:36Z")
I think the option to do that from the ini file was removed a couple of versions ago. But if you force mul to throw `NotSuitableError` then you’ll fall back on cuBLAS. Throw the error as soon as you get into the function, somewhere around here: [PyFR/gimmik.py at c8c053d5c0e34ac7a5639c4d328ebacde10fe689 · PyFR/PyFR · GitHub](https://github.com/PyFR/PyFR/blob/c8c053d5c0e34ac7a5639c4d328ebacde10fe689/pyfr/backends/cuda/gimmik.py#L15) To force PyFR to use GiMMiK you can comment out the line a bit later that throws that error. ---
**Author:** ![luli](https://avatars.discourse-cdn.com/v4/letter/l/5f9b8f/32.png) [@luli](https://pyfr.discourse.group/u/luli)\ **Post date:** [15 February 2022 06:43 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/5 "2022-02-15T06:43:30Z")
OK, thank you! I also wonder the performance in different hardware like between A100 and A10. When testing the examples provided, it shows that the GPU-util is very low ![image](https://global.discourse-cdn.com/free1/uploads/pyfr/original/1X/b226519af63038621b1ba59923ccb6b7a860debf.png). Is there anything wrong ? ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [15 February 2022 07:28 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/6 "2022-02-15T07:28:14Z")
That GPU utility number isn’t very informative about how a task is performing. If you want to understand if a program is performing better I suggest using NVIDIA night compute (`ncu`), you might have to ask your system administrator for some additional permissions in order to use it. With ncu you’ll be able to see the bandwidth and flop numbers for each kernel. This should give you a better idea of how it is performing. In my experience however, the performance with PyFR and GiMMiK on A100s is about where I would expect it. Although we are working on ways to make more use of some of the new features of the A100, such as async mem copies, that bypass registers when loading into shared. ---
**Author:** ![fdw](https://avatars.discourse-cdn.com/v4/letter/f/a587f6/32.png) [@fdw](https://pyfr.discourse.group/u/fdw)\ **Post date:** [15 February 2022 12:47 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/7 "2022-02-15T12:47:09Z")
As noted in the _Examples_ section of the documentation > PyFR includes several test cases to showcase the functionality of the solver. **It is important to note, however, that these examples are all relatively small 2D simulations and, as such, are _not_ suitable for scalability or performance studies.** Regards, Freddie. ---
**Author:** ![luli](https://avatars.discourse-cdn.com/v4/letter/l/5f9b8f/32.png) [@luli](https://pyfr.discourse.group/u/luli)\ **Post date:** [16 February 2022 03:01 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/8 "2022-02-16T03:01:41Z")
Thank you! Could you please provide some relatively large examples simulating with new verision PyFR? ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [16 February 2022 07:37 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/9 "2022-02-16T07:37:09Z")
Many of the papers by the PyFR team will include meshes in the supplementary material, which is a good place to look. If you are trying to measure performance, this topic might be a good place to start: [TGV Performance Numbers](https://pyfr.discourse.group/t/tgv-performance-numbers/407) I think I link to a reasonably sized cube mesh that you can try. ---
**Author:** ![luli](https://avatars.discourse-cdn.com/v4/letter/l/5f9b8f/32.png) [@luli](https://pyfr.discourse.group/u/luli)\ **Post date:** [9 May 2022 07:08 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/10 "2022-05-09T07:08:46Z")
Is it cublas or cutlass that pyfr uses when calling the matrix multiplication math library? ---
**Author:** ![WillT](https://avatars.discourse-cdn.com/v4/letter/w/bc79bd/32.png) [@WillT](https://pyfr.discourse.group/u/WillT)\ **Post date:** [9 May 2022 07:48 UTC](https://pyfr.discourse.group/t/whats-the-relationship-between-gimmik-and-cublas/550/11 "2022-05-09T07:48:30Z")
cublas, see here: [PyFR/cublas.py at d175beccd3fc5587903cc00c0f401041ff22abe4 · PyFR/PyFR · GitHub](https://github.com/PyFR/PyFR/blob/d175beccd3fc5587903cc00c0f401041ff22abe4/pyfr/backends/cuda/cublas.py)