Stitching images from a conventional camera and a fisheye camera based on nonrigid warping

Abstract

Conventional cameras and fisheye cameras are often used together to capture clear target images and large scene background images in many applications, such as mobile robotic telepresence systems and large scene monitoring systems. In this paper, we propose to stitch images from these cameras for offering remote operators a large field of view to perceive a local environment. To provide a clear view of targets for face-to-face communication and a complete view of a robot’s surroundings for safe teleoperation of the robot, we stitch these images by keeping the original conventional image. The image stitching is formulated as a nonrigid motion estimation problem and images are stitched based on nonrigid warping, e.g., the thin-plate spline. To improve the algorithmic efficiency of image stitching, we exploit a region-based point correspondence selection method to reduce the number of point correspondences that are used for thin-plate spline interpolation. The experiments conducted on collected images and images captured from a telepresence system show the effectiveness of the proposed method.