Beyond Clear Paths: Training with Neural Style Transfer and Auto-Augmentation for Domain-Generalized Segmentation of Soiled Images

Abstract

The use of image segmentation within mission-critical autonomous vehicle applications is contingent on reliability of visual sensors, which may become significantly compromised by environmental occlusions such as mud, water, dust, or ice. This thesis explores semantic segmentation techniques tailored for identifying occluded regions, addressing the limitations of conventional methods under unpredictable real-world conditions. Given the scarcity of training datasets with realistic occlusions and the resulting reliance on synthetic data, this research leverages style transfer augmentations and a novel augmentation policy optimization framework to diversify training datasets and improve model generalization. A newly-curated dataset of naturally occurring occlusions evaluates the efficacy of these augmentation methods to out-of-domain image samples. By combining empirical experimentation with theoretical insights into domain generalization, this work presents a robust approach to enhancing segmentation performance while investigating model generalization outcomes after training under a variety of occluded environments.