Content area
Various machine learning algorithms can benefit from exploiting the metric structure inherent in data and model parameters. Such geometric structure can be a useful prior to reduce sample complexity, improve generalization, and ensure feasibility of learned models. We present a range of metric-informed machine learning methods that span probabilistic graphical models, reinforcement learning, and sampling from probability distributions.
Within the domain of probabilistic graphical models, we use metric structure to learn feasible quantum-graphical models using geometry-aware optimization. Specifically, we present an approach to learn hidden quantum Markov models (HQMMs) through Riemannian gradient descent on the Stiefel manifold. Moreover, we establish a hierarchy of expressiveness relationships between HQMMs and linear sequential systems from various fields including machine learning, formal language, and quantum information.
Within the domain of sampling based probabilistic inference, we use metric structure to define geometry-aware similarity measures for sampling in structured data spaces. Specifically, we adapt the kernel herding algorithm to non-Euclidean spaces using geometry-aware kernels and Riemannian optimization. We demonstrate through empirical evaluations that Riemannian kernel herding outperforms various common heuristics and other geometry-informed approaches.
Within the domain of reinforcement learning (RL), we use metric structure to define geometry-aware representations of states and policies. Specifically, we introduce Behavioral Eigenmaps (BeigeMaps) – state representations that preserve the local metric structure induced by bisimulation metrics through neural eigendecompositions of similarity kernels. When added as a drop-in modification, BeigeMaps improve the policy performance of prior behavioral distance-based RL algorithms by highlighting value-based clusters in state spaces. Moreover, we also introduce a framework for policy composition to aggregate policies learned in multi-objective RL as policy-centroids in distance spaces where policies are embedded.