Content area
Abstract
We pursue local sparse representations of data by considering a common data model where representations are formed as a combination of atoms that we call a dictionary. Our focus is on two paradigms of the model: 1) linear representations where data are treated as linear combinations of the atoms, and 2) non-linear representations where we treat data and the atoms as measures. Our work can be contextualized within the specific areas of sparse representations and dictionary learning, where we seek to find descriptions of the data using as few dictionary atoms as necessary, which is why we call them sparse representations. We then arrive at local sparse representations from a further desire to have the few atoms used lie close to the data being represented. Unifying our work in Euclidean and Wasserstein spaces is a geometric regularizer that encourages the use of local atoms.
In the Euclidean case, we focus on solving a regularized optimization problem to find the weights that represent a given data point as a convex combination of a given dictionary. We show how the use of this regularizer, based on a Euclidean locality function, identifies sparse solutions that form a local representation of the data point. We prove that this notion of local sparsity is grounded with connections to the Delaunay triangulation.
The more general problem of dictionary learning is considered in Wasserstein space. Now, representations are formed as Wasserstein barycenters; a non-linear method of averaging measures. Wasserstein dictionary learning seeks a dictionary of measures that can thus be used to represent the data as barycenters thereof. We extend the existing framework with the inclusion of a geometric regularizer to the objective that promotes forming representations with atoms that are close to the data to represent in Wasserstein distance. We demonstrate how our regularizer promotes unique, interpretable, sparse solutions that are useful for downstream tasks such as classification.