A Deep Q-Learning Based UAV Detouring Algorithm in a Constrained Wireless Sensor Network Environment

Abstract

Unmanned aerial vehicles (UAVs) play a crucial role in various applications, including environmental monitoring, disaster management, and surveillance, where timely data collection is vital. However, their effectiveness is often hindered by the limitations of wireless sensor networks (WSNs), which can restrict communications due to bandwidth constraints and limited energy resources. Thus, the operational context of the UAV is intertwined with the constraints on WSNs, influencing how they are deployed and the strategies used to optimize their performance in these environments. Considering the issues, this paper addresses the challenge of efficient UAV navigation in constrained environments while reliably collecting data from WSN nodes, recharging the sensor nodes’ power supplies, and ensuring the UAV detours around obstacles in the flight path. First, an integer linear programming (ILP) optimization problem named deadline and obstacle-constrained energy minimization (DOCEM) is defined and formulated to minimize the total energy consumption of the UAV. Then, a deep reinforcement learning-based algorithm, named the DQN-based UAV detouring algorithm, is proposed to enable the UAV to make intelligent detour decisions in the constrained environment. The UAV must finish its tour (data collection and recharging sensors) without exceeding its battery capacity, ensuring each sensor has the minimum residual energy and consuming energy for transmitting and generating data, after being recharged by the UAV at the end of the tour. Finally, simulation results demonstrate the effectiveness of the proposed DQN-based UAV detouring algorithm in data collection and recharging the sensors while minimizing the total energy consumption of the UAV. Compared to other baseline algorithm variants, the proposed algorithm outperforms all of them.