HotQCD on multi-GPU Systems

We present $\texttt{SIMULATeQCD}$, HotQCD’s software for performing lattice QCD calculations on GPUs. Started in late 2017 and intended as a full replacement of the previous single GPU lattice QCD code used by the HotQCD collaboration, our software has been developed into an extensive framework for lattice QCD calculations distributed on multiple GPUs over many compute nodes. The code is built on C++, CUDA, and MPI and leverages modern C++ language features to provide high-level data structures, objects, and algorithms that allow users to express lattice QCD calculations in an intuitive way without sacrificing performance. Implemented algorithms range from gradient flow, correlator measurements, and mixed precision conjugate gradient solvers all the way to full HISQ gauge field configuration generation using RHMC. After successful deployment in large-scale computing projects, we want to share the result of our efforts with the lattice QCD community by making it publicly available. In these proceedings, we will present some of the key features of our code, demonstrate its ease of use, and show benchmarks of performance critical kernels on state-of-the-art supercomputers.