Improving Robustness of Perception DNNs in Various Domains

In recent years, deep learning has gained tremendous popularity for solving various challenges in highly automated driving, such as environment perception, sensor fusion, motion planning, etc. In this context, semantic segmentation appears to be a vital task that helps identify objects in a scene around the car. However, deep learning models frequently struggle with robustness concerning diverse input distribution shifts, including sensor noise, photometric changes, motion blur, and adversarial attacks. This lack of robustness poses a substantial safety challenge that must be addressed to ensure the reliability and safety of highly automated driving systems.

This thesis focuses on the lack of robustness issue in camera-based semantic segmentation tasks. We begin by benchmarking two state-of-the-art semantic segmentation models for their resilience to input corruptions and adversarial attacks. The benchmarking uncovers robustness issues in both models, with adversarial attacks inflicting more damage than corruptions. We then aim to enhance these models' robustness by proposing a variety of methods.

We make three contributions towards improving adversarial robustness. First, we explore various frequency-domain adversarial attacks and detect strong systematic artifacts in adversarial images, which are absent in clean images. Detailed investigations reveal these artifacts are agnostic with respect to the attack type and setting and largely depend on the network architecture. More specifically, we report a closer connection between the interpolation type used in the network layers and the resulting artifacts. Second, we devise a novel input pre-processor-based adversarial defense method using Wiener filters to suppress these artifacts, effectively defending against multiple adversarial attacks and outperforming five other classical denoising pre-processor-based defenses. Third, we propose a more sophisticated pre-processor-based defense employing a vector-quantized variational autoencoder (VQVAE), which surpasses the Wiener filter-based defense and comparable autoencoder-based methods.

To enhance robustness against natural corruptions, we introduce a novel self-supervised regularization loss function that boosts corruption robustness when combined with state-of-the-art data augmentation methods like AugMix. This training method can be seamlessly integrated with our proposed input pre-processors to defend against both adversarial attacks and corruptions simultaneously. Furthermore, we rigorously test our proposed methods under real-world adverse conditions, such as fog, snow, night, and rain, and report substantial improvements.

In summary, the proposed methods pave a way to train robust semantic segmentation models capable of handling diverse unseen input distributional shifts, including adversarial attacks and natural corruptions. Detailed evaluations and research directions are presented for each method and open the door to improved safety and reliability of autonomous systems in complex environments.