Virtually Anywhere: Robust Pose Acquisition and Tracking for XR With Reduced Reliance on Vision and Infrastructure

Abstract

In order for XR interfaces to evoke the illusion of virtual content coexisting with the physical world, they rely on precise localization and tracking to spatially align the two. Current platforms accomplish this through a myriad of sensing, estimation, and mapping approaches, ranging from lightweight tracking with inexpensive cameras (eg. ARKit, Snapchat, Meta Quest) to full-fledged optical motion capture systems (eg. Hollywood VFX studios). However, existing platforms are unsuitable for applications that require instant-on and/or infrastructure-free operation, particularly in domains where vision-based sensors fall short. In this work, we address these limitations by exploring active visual fiducials, lightweight radio-frequency infrastructure, peer-to-peer localization, and radar-based tracking and mapping. We show how these techniques can drastically improve the reliability of pose acquisition and tracking for XR platforms while simultaneously reducing infrastructure costs, with the goal of enabling truly ubiquitous spatial understanding for future XR applications.