GPU Support

GPU support is still an experimental feature that is actively being worked on. As of now, the WeaklyCompressibleSPHSystem and the BoundarySPHSystem are supported on GPUs. We have tested this on GPUs by Nvidia and AMD.

To run a simulation on a GPU, we need to use the FullGridCellList as cell list for the GridNeighborhoodSearch. This cell list requires a bounding box for the domain, unlike the default cell list, which uses an unbounded domain. For simulations that are bounded by a closed tank, we can use the boundary of the tank to obtain the bounding box as follows.

search_radius = TrixiParticles.compact_support(smoothing_kernel, smoothing_length)
min_corner = minimum(tank.boundary.coordinates, dims=2) .- search_radius
max_corner = maximum(tank.boundary.coordinates, dims=2) .+ search_radius
cell_list = TrixiParticles.PointNeighbors.FullGridCellList(; min_corner, max_corner)

We then need to pass this cell list to the neighborhood search and the neighborhood search to the Semidiscretization.

semi = Semidiscretization(fluid_system, boundary_system,
                          neighborhood_search=GridNeighborhoodSearch{2}(; cell_list))

At this point, we should run the simulation and make sure that it still works and that the bounding box is large enough. For some simulations where particles move outside the initial tank coordinates, for example when the tank is not closed or when the tank is moving, an appropriate bounding box has to be specified.

Then, we only need to specify the data type that is used for the simulation. On an Nvidia GPU, we specify:

using CUDA
ode = semidiscretize(semi, tspan, data_type=CuArray)

On an AMD GPU, we use:

using AMDGPU
ode = semidiscretize(semi, tspan, data_type=ROCArray)

Then, we can run the simulation as usual. All data is transferred to the GPU during initialization and all loops over particles and their neighbors will be executed on the GPU as kernels generated by KernelAbstractions.jl. Data is only copied to the CPU for saving VTK files via the SolutionSavingCallback.