Abstract

With the increasing deployment of Convolutional Neural Networks (CNNs) in mission-critical and safety-critical applications, ensuring fault tolerance during model inference has become a critical requirement. These applications often rely on Graphics Processing Units (GPUs) to deliver the high computational throughput necessary for fast and efficient inference. However, as GPUs integrate more Processing Elements (PEs) onto a chip and as manufacturing processes continue to scale down, they become increasingly susceptible to hardware faults, including both transient and permanent faults. Furthermore, the massive number of PEs in a GPU makes traditional redundancy-based fault-tolerant techniques impractical due to their significant area and performance overheads.

This paper proposes a low-cost fault-tolerance method called \textit{Sparsity-aware Fault Tolerance (SparseFT)} to ensure reliable CNN inference on GPUs. SparseFT takes advantage of the inherent sparsity in activation and weight maps to detect and correct errors in PEs without requiring additional hardware redundancy. In dot-product computations, multiplications involving zero operands are ineffectual as they result in zero and do not contribute to the final dot-product value. By leveraging this characteristic, SparseFT dynamically duplicates effective computations (i.e., multiplications involving non-zero operands) onto PEs initially allocated for ineffectual computations. It then compares the results of the duplicated computations to detect errors. When an error is detected, SparseFT re-executes the erroneous computation on error-free PEs to correct the fault. If the error is determined to be caused by a permanent fault, SparseFT isolates the faulty PE to prevent it from being used in future operations. This paper also introduces a hardware-software co-design that incorporates novel pruning techniques to enhance SparseFT’s error detection coverage. Experimental results demonstrate that SparseFT achieves over 99% error detection coverage with less than 1% performance overhead across state-of-the-art CNN models.