Is there any tutorial teaching how to use ascent in a MPI+cuda structured mesh code?
lj-cug opened this issue · 4 comments
As the title of this issue, I need the tutorial code to use ascent in my hydrodynamic code. Thanks!
@lj-cug we don't have a tutorial specific to this, but if you build with CUDA support Ascent will use CUDA and you can explore the cloverleaf app as MPI simulation that provides rectilinear grid. We don't have a structured mesh example (explicit coordinates with implicit topology) wired up in our examples.
MandelbrotCatalyst2 code snippet (https://github.com/jspanchu/MandelbrotCatalyst2) converts conduit mesh blueprints into the VTK data model – vtkConduitSource, such that it can ingest data pointers from external memory spaces (CUDA device memory).
MandelbrotCatalyst2 demo code seamlessly send pointers related to mesh coordinates, connectivity, and fields directly to vtkConduitSource. Simulations can send these pointers by populating nodes in the conduit mesh blueprint.
Finally, the ParaView catalyst implementation was updated to accommodate the new memory space identifier and configure vtkConduitSource for external memory spaces.
I cannot search a CUDA code showing how to use Ascent API. MandelbrotCatalyst2 demo code is for the single-GPU to enable GPU zero copy between CUDA and ParaView-Catalyst2. I have also read the proxies/cloverleaf3d-ref.
And the MPI-CUDA version of cloverleaf use CUDA kernels, the data is copied from CUDAto CPU host to export data, I don't know how to use Ascent in there kernels.
So if Ascent developer can supply a proxies using pure CUDA code like MandelbrotCatalyst2, this will help me.
Thanks
@lj-cug Thanks, yes an example of a CUDA simulation would be helpful to teach this, but we don't have one yet. (We do have a ticket for this #597)
In Ascent we accept pointers that are already on the device via Conduit.
When they are passed to VTK-m internally, we ask where they are (host or GPU) and support moving them to the GPU when needed.
Our goal is to not need any extra annotations about where the data is resident in memory.
There may be a few places where this assumption fails, but we will work to fix those cases.
The same conduit mesh setup used in the Catalyst example you shared should work with Ascent.
Both the calculation and rendering happened in GPU, this case is seldomly seen. Thank you!