Review on Hardware Devices and Software Techniques Enabling Neural Network Inference Onboard Satellites

Abstract

Neural networks (NNs) have proven their ability to deal with many computer vision tasks, including image-based remote sensing such as the identification and segmentation of hyperspectral images captured by satellites. Often, NNs run on a ground system upon receiving the data from the satellite. On the one hand, this approach introduces a considerable latency due to the time needed to transmit the satellite-borne images to the ground station. On the other hand, it allows the employment of computationally intensive NNs to analyze the received data. Low-budget missions, e.g., CubeSat missions, have computation capability and power consumption requirements that may prevent the deployment of complex NNs onboard satellites. These factors represent a limitation for applications that may benefit from a low-latency response, e.g., wildfire detection, oil spill identification, etc. To address this problem, in the last few years, some missions have started adopting NN accelerators to reduce the power consumption and the inference time of NNs deployed onboard satellites. Additionally, the harsh space environment, including radiation, poses significant challenges to the reliability and longevity of onboard hardware. In this review, we will show which hardware accelerators, both from industry and academia, have been found suitable for onboard NN acceleration and the main software techniques aimed at reducing the computational requirements of NNs when addressing low-power scenarios.