An augmented reality maintenance assistant with real-time quality inspection on handheld mobile devices

Abstract

Advances of industry 4.0 are enabling augmented reality (AR) devices to be deployed across the manufacturing sector to enhance worker engagement and performance. This evolution of AR in the workspace is driving changes in how workers on the factory floor interact with the enterprise. AR is evolving to provide work instructions for assembly and maintenance. Recent demonstrations show the capability to allow for quality inspection to be integrally evaluated as part of the AR workflow. Previous integrations of quality include the ability for manual assessment to be logged into the enterprise as part of completing work instructions; however, this is largely based on the perception of the user to determine if they accurately performed the AR instructions. This study is designed to demonstrate the ability of an enterprise-level AR maintenance experience to go beyond manual inspection by embedding real-time quality assessment into the work instructions. Essentially, the goal is to enable the enterprise to perform a self-assessment of the quality of the work performed by maintenance personnel. Furthermore, the AR instructions enable additional instruction to empower maintenance personnel to correct their work prior to moving on to other tasks. The novelty of this work is focused on deployment on a mobile device with integrated vision inspection using the same device that is being used to deploy the AR work instructions. This work enumerates several essential qualities of mobile quality inspection tools and outlines some of the challenges associated with the development of such a system. Finally, results from testing the system demonstrate that handheld mobile devices can be used to capture inspection images while simultaneously deploying AR work instructions. Analysis of the image correction, timing for vision process of the quality check, and overall performance of the integrated system are presented.