Publications

HopCraft: Hotness-Aware Dynamic Clustering in Wafer-Scale GPUs

IEEE Computer Architecture Letters

Card image

Abstract

Wafer-scale GPUs integrate many GPU modules (GPMs) on a single wafer to meet the high compute demand of modern workloads. However, a larger GPM count forces remote memory accesses to traverse longer paths, increasing the average memory access latency. In this paper, we propose \textit{HopCraft} to mitigate the overhead of remote accesses in wafer-scale GPUs. HopCraft identifies hot data at the instruction level and replicates it within clusters of GPMs whose size adapts to the hotness and working set of the data. As a result, hot data is placed at suitable locations on the wafer, and remote requests can be served from a nearer copy. Our evaluation shows that HopCraft improves performance by 41% and reduces average memory access latency by 35% compared to NearFetch, the state-of-the-art technique.