Highly overparameterized optical flow using PatchMatch belief propagation

TitleHighly overparameterized optical flow using PatchMatch belief propagation
Publication TypeConference Paper
Year of Publication2014
AuthorsHornáček, M, Besse, F, Kautz, J, Fitzgibbon, A, Rother, C
Conference NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISBN Number9783319105772
Keywords9 DoF, large displacement, optical flow, PatchMatch, PMBP

Motion in the image plane is ultimately a function of 3D motion in space. We propose to compute optical flow using what is ostensibly an extreme overparameterization: depth, surface normal, and frame-to-frame 3D rigid body motion at every pixel, giving a total of 9 DoF. The advantages of such an overparameterization are twofold: first, geometrically meaningful reasoning can be called upon in the optimization, reflecting possible 3D motion in the underlying scene; second, the 'fronto-parallel' assumption implicit in the use of traditional matching pixel windows is ameliorated because the parameterization determines a plane-induced homography at every pixel. We show that optimization over this high-dimensional, continuous state space can be carried out using an adaptation of the recently introduced PatchMatch Belief Propagation (PMBP) energy minimization algorithm, and that the resulting flow fields compare favorably to the state of the art on a number of small- and large-displacement datasets. © 2014 Springer International Publishing.

Citation KeyHornacek2014