MCS-Sim: A Photo-Realistic Simulator for Multi-Camera UAV Visual Perception Research

Abstract

Multi-camera systems (MCSs) are pivotal in aviation surveillance and autonomous navigation due to their wide coverage and high-resolution sensing. However, challenges such as complex setup, time-consuming data acquisition, and costly testing hinder research progress. To address these, we introduce MCS-Sim, a photo-realistic MCS simulator for UAV visual perception research. MCS-Sim integrates vision sensor configurations, vehicle dynamics, and dynamic scenes, enabling rapid virtual prototyping and multi-task dataset generation. It supports dense flow estimation, 3D reconstruction, visual simultaneous localization and mapping, object detection, and tracking. With a hardware-in-loop interface, MCS-Sim facilitates closed-loop simulation for system validation. Experiments demonstrate its effectiveness in synthetic dataset generation, visual perception algorithm testing, and closed-loop simulation. Here we show that MCS-Sim significantly advances multi-camera UAV visual perception research, offering a versatile platform for future innovations.