Improving parallelism of scientific and engineering applications on heterogeneous supercomputers

Abstract

The rising usage of heterogeneous supercomputers introduces both opportunities for increased parallelism and challenges for efficient usage of the available hardware. Applications running on heterogeneous supercomputers must adopt new methods to achieve performance across two levels of parallelism. Inter-process parallelism defines coordination between processes and intra-process parallelism within each process. This thesis presents research towards improving inter-process and intra-process parallelism for applications that use complex data structures such as distributed unstructured meshes. Inter-process parallelism is defined by the coupled costs of the partition of load between processes and the communications between processes required as a result of the partition. To achieve optimal performance, partitions must divide computational load evenly between processes while minimizing the additional costs of communications. This thesis addresses improving inter-process parallelism using multicriteria partition improvement multicriteria methods on a generalized structure for a broad set of potential applications. The partition improvement methods are applied to different unstructured mesh setups with partitions up to half a million processes. In the case of heterogeneous supercomputers, intra-process parallelism is dictated by the parallel hardware available to each process for performing computations. For most of the current and next generation US systems, Graphic Processing Units (GPUs) are the parallel hardware available on each node. This thesis addresses methods for intra-process parallelism in the scope of particle-in-cell simulations with a novel approach to the storage of the unstructured mesh and the particles for optimized performance on GPUs while utilizing performance-portable methods for performance on future hardware. Scaling studies of these methods are presented up to 4096 nodes of the Summit supercomputer with over a trillion particles simulated.