Dynamic sparse coding for sparse time-series modeling via first-order smooth optimization

Abstract

Sparse coding, often called dictionary learning, has received significant attention in the fields of statistical machine learning and signal processing. However, most approaches assume iid data setup, which can be easily violated when the data retains certain statistical structures such as sequences where data samples are temporally correlated. In this paper we formulate a novel dynamic sparse coding problem, and propose an efficient algorithm that enforces smooth dynamics for the latent state vectors (codes) within a linear dynamic model while imposing sparseness of the state vectors. We overcome the added computational overhead originating from smooth dynamic constraints by adopting the recent first-order smooth optimization technique, adjusted for our problem instance. We demonstrate the improved prediction performance of our approach over the conventional sparse coding on several interesting real-world problems including financial asset return data forecasting and human motion estimation from silhouette videos.