3D Full-body Pose Estimation of Construction Machines Using Deep Neural Network and Stereo Vision

Abstract

The movement and posture variability of construction machines is a significant contributor to safety hazards on construction sites. Even when a machine's location is fixed, its moving parts may collide with on-site personnel or objects, leading to injuries or production loss. Accurate estimation of 3D full-body poses of machines can enhance safety by providing more precise spatial information. This paper proposes a framework to estimate 3D full-body poses of construction machines using deep neural networks (DNNs) and stereo vision. The proposed framework employs an entropy-based active learning method to select informative images for fine-tuning the DNN model for 2D pose estimation. 3D poses are estimated through stereo camera calibration, coarseto-fine stereo matching, and triangulation. Experimental validation using an excavator model achieved an average error percentage (AEP) of 12.11%, demonstrating the framework's feasibility for enhancing safety management.