Multi-Armed Bandit Approaches for Location Planning with Dynamic Relief Supplies Allocation Under Disaster Uncertainty

Abstract

Highlights

What are the main findings?

• We design a three-layer emergency logistics network to manage the flow of disaster relief materials and develop a bi-objective, multi-period stochastic integer programming model to support post-disaster decision-making under uncertainty. Multi-armed bandit approaches are innovatively applied to solve the problem.

• A newly developed multi-armed bandit (reinforcement learning) algorithm called the Geometric Greedy algorithm, achieves overall higher performance than the traditional ϵ-Greedy algorithm and the Upper Confidence Bound (UCB) algorithm.

What is the implication of the main finding?

• The key advantage of using reinforcement learning to solve our problem is that agents can dynamically adjust their strategies through interaction with the uncertain environment to minimize action costs.

Abstract

Natural disasters (e.g., floods, earthquakes) significantly impact citizens, economies, and the environment worldwide. Due to their sudden onset, devastating effects, and high uncertainty, it is crucial for emergency departments to take swift action to minimize losses. Among these actions, planning the locations of relief supply distribution centers and dynamically allocating supplies is paramount, as governments must prioritize citizens’ safety and basic living needs following disasters. To address this challenge, this paper develops a three-layer emergency logistics network to manage the flow of emergency materials, from warehouses to transfer stations to disaster sites. A bi-objective, multi-period stochastic integer programming model is proposed to solve the emergency location, distribution, and allocation problem under uncertainty, focusing on three key decisions: transfer station selection, upstream emergency material distribution, and downstream emergency material allocation. We introduce a multi-armed bandit algorithm, named the Geometric Greedy algorithm, to optimize transfer station planning while accounting for subsequent dynamic relief supply distribution and allocation in a stochastic environment. The new algorithm is compared with two widely used multi-armed bandit algorithms: the $ϵ$-Greedy algorithm and the Upper Confidence Bound (UCB) algorithm. A case study in the Futian District of Shenzhen, China, demonstrates the practicality of our model and algorithms. The results show that the Geometric Greedy algorithm excels in both computational efficiency and convergence stability. This research offers valuable guidelines for emergency departments in optimizing the layout and flow of emergency logistics networks.