Variational Recursive Joint Estimation of Dense Scene Structure and Camera Motion from Monocular Image Sequences

Principal investigators: Florian Becker, Frank Lenzen, Jörg H. Kappes, Christoph Schnörr

Overview

We present an approach to jointly estimating camera motion and dense structure of a static scene in terms of depth maps from monocular image sequences in driver-assistance scenarios. At each instant of time, only two consecutive frames are processed as input data of a joint estimator that fully exploits second-order information of the corresponding optimization problem and effectively copes with the non-convexity due to both the imaging geometry and the manifold of motion parameters. Additionally, carefully designed Gaussian approximations enable probabilistic inference based on locally varying confidence and globally varying sensitivity due to the epipolar geometry, with respect to the high-dimensional depth map estimation. Embedding the resulting joint estimator in an online recursive framework achieves a pronounced spatio-temporal filtering effect and robustness.

For questions, suggestions and further information, please contact Two consecutive frames (size: 656 x 541 pixels) of the Bend image sequence recorded by a fast moving camera. Displacements of up to 35 pixels can be observed. Click to enlarge the images. Left: Our approach jointly estimates the scene structure represented by a dense depth map (visualized using a non-linear color map) and the camera motion in an online recursive framework. Right: reconstruction of the scene structure based on the depth map from the camera's viewpoint (green symbol) as well as the camera track (red line). Click to enlarge the images. Further information and results can be found in Becker et al. (2013) (an extension of Becker et al (2011)) - and the corresponding supplemental material.

Acknowledgments

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.