Interoceptive Robotic Agents for Shared Control and Learning in Collaborative ConstructionWork

Abstract

Advancing the adaptivity of autonomous mobile robots (AMRs), particularly empowering them to align with human expertise and learn from past experiences to infer optimal decisions in response to changing task demands and dynamic environment, is critical for their application in modular construction automation and reducing operational carbon footprints. Although a few learning paradigms based on human demonstrations have been introduced, robots remain rigid in structured tasks and lack internal adaptivity and authentic learning capabilities. This research explores a fundamental approach to equipping robots with an internal drive for self-reflection and conscious learning within a shared control setting, where human supervisors provide occasional corrective inputs and interventions. This approach is centered on the concept of interoception, which we factorize as "cognitive dissonance" within the robotic cognitive architecture. When significant decision divergence with human counterparts occurs, the robotic agent will actively seek decision rationale from semantic inputs, extracts knowledge from human scaffolding, encodes it into a memory system, and retrieves this "cached" knowledge for future decision-making in similar scenarios. This paper explores this new perspective to help build adaptive motion planning in AMRs. First, we propose integrating the legacy of heuristic costs from grid/graphbased path planning algorithms with a hypergraph model that caches declarative and procedural knowledge extracted from human semantic inputs. Second, we design a velocityreplay module that uses few-shot contrastive learning with a encoder-decoder architecture and compare its performance against reinforcement learning with human feedback and imitation learning. Two simulation demonstrations were conducted using the Issac Sim platform to showcase multi-robot synchronization and stacking collaboration. In addition, the proposed shared control paradigm was tested in a Gazebo environmentin ROS, where human inputs were provided via relatively noisy brainwave-based control, compared to conventional peripheral devices. The results demonstrate how this approach better mitigates human input errors while facilitating autonomy for task completion compared to existing dominant shared control methods. The insights from this study resonate with recent advances in neuroscience and reinforcement learning, and pave the way toward artificial general intelligence in AMRs, fostering their progression from complexity to competence in construction automation.