High-performance GPU techniques for 2D convolution operators in deep neural networks

This paper presents an optimized implementation of Winograd non-fused convolution. Our optimizations include both application independent and Winograd-specific software techniques, such as a specialized interface-kernel data format (tile-united CNHW layout) to enhance memory access efficiency; warp specialization and double-buffered prefetching to effectively exploit computational resources and memory bandwidth; and the use of shuffle instructions to conserve hardware resources. We propose a GPU-based Multi-Modal Parallelism Method (MMPM) for 2D Winograd non-fused convolution and provide a supplementary explanation of Winograd’s tile extraction, which
reduces memory usage and computation. The proposed techniques were evaluated at the kernel level in two environments (ENV1 – GTX 980 GPU, CUDA 9.2, cuDNN 7.6.4; ENV2 – GTX 1650Ti GPU, CUDA 10.2, cuDNN 8.2.0) using a wide range of CNN layer benchmark-compliant parameters. Compared with the state-of-the-art Winograd non-fused convolution in cuDNN, our implementation achieves speedups of 1.64× and 1.28× for the two environments, respectively.