Online 3D Bin Packing An Image-Based Deep Reinforcement Learning Approach

Abstract

The bin-packing problem is a strongly NP-Hard problem with extensive research. It involves the task of arranging a set of items into a finite number of bins and trying to optimize against some sort of heuristic, usually involving maximizing the number of items placed or minimizing the amount of empty space. The online case involves placing items without information on the upcoming sequence of items. It has significant applications in warehouse management, e-commerce logistics, and cloud computing. In this paper, we explore an image-based approach using deep reinforcement learning to teach a model to place geometric items efficiently in the 3D case. Image based techniques have the benefit of not requiring precise measurements of the bin state or object being placed, and can also represent non-uniform shapes easier. We leverage a Double Deep Q Learning network as our deep reinforcement learning framework to teach a model to place an item given an image of the bin state as well as the item to place. We use a reward structure defined in a manner to encourage compactness and clustering of the items, as well as discouraging overlapping / out of bounds invalid moves. The results show that we outperform the baseline heuristics and compete with state-of-the-art methods for 3D online bin packing when using small bin dimensions.