Container loading planning using reinforcement learning based on curriculum learning

Abstract

Planning container loading is essential for cost-effective efficiency improvements in port logistics systems. Currently, load planning is performed either manually or semi-automatically. However, as the trend of ultra-large containers continues, manually calculating an efficient loading plan incurs high costs. To solve this problem, many studies have been conducted by considering factors such as container weight, unloading order, and balance. However, existing studies show that when the bay or the number of containers to be loaded changes, a new model must be retrained or recalculated, which incurs high costs. Therefore, this study proposes a container loading plan that can quickly adapt to environmental changes. A curriculum technique was used to create an environment ranging from easy to complex. The loading plan was conducted using the proximal policy optimization algorithm, which has a fast convergence speed among reinforcement learning algorithms. The efficiency of this study was verified by comparisons with the methodology used in existing studies.