A Unified Frequency Understanding of Image Corruptions and its Application to Autonomous Driving 2023-01-0060

Image corruptions due to noise, blur, contrast change, etc., could lead to a significant performance decline of Deep Neural Networks (DNN), which poses a potential threat to DNN-based autonomous vehicles. Previous works attempted to explain corruption from a Fourier perspective. By comparing the absolute Fourier spectrum difference between corrupted images and clean images in the RGB color space, they regard the noise from some corruptions (Gaussian noise, defocus blur, etc.) as concentrating on the high-frequency components while others (contrast, fog, etc.) concentrate on the low-frequency components. In this work, we present a new perspective that unifies corruptions as noise from high frequency and thus propose an image augmentation algorithm to achieve a more robust performance against common corruptions. First, we notice the 1/f^α statistical rule of the natural image's spectrum and the channels-wise differential sensitivity on the YCbCr color space of the Human Visual System. Thus we present a new perspective of the relative Fourier spectrum in the YCbCr color space which unifies the noise from all 15 common corruptions as noise concentrating on the high-frequency components. From the new perspective, the SOTA (state-of-the-art) image augmentation algorithm shows insufficient coverage of frequency change compared with common corruptions and an insufficient improvement in robustness against perturbations on the frequency domain. Second, based on the unified understanding of image corruptions and limitations of the SOTA image augmentation algorithm, we present a frequency-based image augmentation that adds noise to the high-frequency components in the YCbCr color space with the amplitude of noise proportional to the spectrum amplitude of the image. Compared with previous works, the proposed method achieves SOTA performance against corruptions on common datasets (CIFAR-10 and tiny-ImageNet) and real-world driving tests.