AutoMergeNet: AutoML-Based M-Source Satellite Data Fusion Evaluated With Atmospheric Case Studies

Accurate detection of anomalous phenomena in satel lite data often requires data layers containing complementary information (e.g., data from different sensors, auxiliary features, such as land cover maps, and metadata regarding data quality). However, existing highly specialized approaches to fuse multiple data layers cannot be transferred to other related problems, as they rely on expert-selected features and manual pipeline design. In this work, we propose AutoMergeNet, a framework for satellite image data fusion based on neural architecture search. AutoMer geNet generates neural networks that fuse any number of raster data layers. Consequently, it can address different classification problems based on satellite images without manual pipeline design. We designed the search space of AutoMergeNet by identifying relevant design choices from the image classification and data fusion literature. AutoMergeNet automatically transforms image classification networks into multibranch networks by optimizing critical architectural and training hyperparameters. Since the high dimensionality of multimodal image data poses a challenge for data fusion problems with limited labels, we use an auxiliary unimodal classifier combined with AutoMergeNet. We evaluate AutoMer geNet on a methane plume detection dataset from the literature and our newly created carbon monoxide plume detection dataset. Au toMergeNet performs strongly and consistently on these two multi modal classification problems, outperforming six baseline methods selected from state-of-the-art image classification approaches. Fi nally, we demonstrate the usability of our framework with a realis tic methane plume detection use case, which shows that AutoMer geNet can be used as a highly specialized, state-of-the-art approach.