Automated methods of visual interpretation: Articulated human tracking in video and segmentation of microscopy images

Abstract

We address two problems related to automated visual interpretation. We first explore automated tracking of a human subject’s pose in a video sequence, a task with many applications from surveillance to human-machine interaction. We propose a novel estimation method for articulated human tracking based on jointly modeling the observed video sequence, the field of background and foreground labels, and the articulated body. We develop a statistical appearance model of the human in a scene and perform graphical inference using nonparametric belief propagation to reliably track a single human in video sequences captured using a single stationary camera.

The second problem we address is segmentation of microscopy images of metal alloys, enabling analysis of materials properties and defects. We explore two segmentation algorithms, each offering novel methods of incorporating a variety of prior knowledge into the segmentation process. These largely automated techniques offer tremendous advantages over manual segmentation conventionally used for alloy analysis.

Our first algorithm is an extension of stabilized inverse diffusion equation (SIDE) segmentation. Our modifications enable the utilization of multiple 2D images to compute a composite segmentation. This data fusion enhances the performance of SIDE beyond that of the original technique which used only a single image. Our second segmentation algorithm offers an approach to incorporate prior knowledge of the shape of microstructures within an alloy. We propose a novel construction shape priors using matching pursuits and implement the priors within a maximum a posteriori (MAP) segmentation framework, solved using a min-cut algorithm.