NUMA-AWARE QUEUE SCHEDULER FOR MULTI-CHIPLET GPUS

Abstract

Chiplet-based architectures have recently emerged as a technique to improve yields and enable continued performance scaling. However, the increased modularity and scalability they offer also requires rethinking system design. Compared to monolithic designs, chiplet-based architectures face challenges around how computation is scheduled and how data movement is coordinated across chiplets. These challenges introduce additional Non-Uniform Memory Access (NUMA) complexities in multi-chiplet systems that can impact performance. Consequently, exploiting lo- cality is a significant bottleneck in multi-chiplet systems. Although multi-chiplet CPUs overcome this inefficiency through complex coherence protocols or OS support, accelerators (e.g., GPUs) utilize relatively lightweight coherence and OS support. Thus, inter-chiplet NUMA effects affect them more – especially at phase boundaries where accelerators often utilize heavyweight oper- ations to ensure correctness. In recognition of these challenges, prior multi-chiplet GPU works introduced mechanisms to improve data locality or reduce synchronization overhead. However, these techniques perform these optimizations in isolation, limiting their benefits. Conversely, we propose CAQS, a novel Cache-Aware Queue Scheduler that intelligently utilizes both locality and synchronization information when deciding where to schedule application phases to reduce the impact of inter-chiplet NUMA effects. Overall, across 18 popular GPU workloads CAQS im- proves geomean performance (30%, 28%, 6%), energy efficiency (36%, 19%, 27%), and reduces network traffic (80%, 61%, 80%), over modern GPUs and the state-of-the-art CPElide and LADM, respectively. Moreover, CAQS’s advantages grow for more concurrent streams.