{"id":1680,"date":"2015-07-29T13:57:01","date_gmt":"2015-07-29T13:57:01","guid":{"rendered":"http:\/\/cvlab-dresden.de\/?page_id=1680"},"modified":"2015-10-19T20:27:24","modified_gmt":"2015-10-19T20:27:24","slug":"graphflow","status":"publish","type":"page","link":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/image-matching\/graphflow\/","title":{"rendered":"GraphFlow"},"content":{"rendered":"

H. Abu Alhaija, A. Sellent, D. Kondermann, C. Rother. “GraphFlow – 6D \u00a0Large Displacement Scene Flow via Graph Matching”, German Conference on Pattern Recognition (GCPR, a.k.a. DAGM), 2015. [pdf<\/a>]<\/span><\/p>\n

We present an approach for computing dense scene flow from two large displacement RGB-D images. When dealing with large displacements the crucial step is to estimate the overall motion correctly. While state-of-the-art approaches focus on RGB information to establish guiding correspondences, we explore the power of depth edges. To achieve this, we present a new graph matching technique that brings sparse depth edges into correspondence. An additional contribution is the formulation of a continuous-label energy which is used to densify the sparse graph matching output. We present results on challenging Kinect images, for which we outperform state-of-the-art techniques.<\/p>\n

Graph Matching<\/h2>\n

In our two stage scene flow approach, instead of using sparse texture matches only, we additionally utilize depth edges extracted from the RGB-D images that describe object boundaries well.However, in the presence of large motion, they are actually not trivially described and matched.While exact edge description suffers from occlusion and distortion effects, more robust edge descriptors often lead to ambiguous matches.\u00a0To disambiguate edge matches with robust descriptors, we use a structured matching approach in the form of graph matching that profits from non-local information to assign edge matches.<\/p>\n

\n

\"Screenshot_080315_030041_PM\"<\/a><\/p>\n

Fig. 1: Details of graph matching. (a) Our edge description segments are\u00a0represented by their center point, average appearance descriptor that describe\u00a0the foreground region and normalized depth gradient vector. In order to compute\u00a0a description segment we accumulate neighboring edge pixels whose descriptor\u00a0variance is lower than a threshold \u001bt and whose count is between rmin = 20\u00a0and rmax = 30 pixels. (b) For graph matching, the description segments centers\u00a0are connected to form a graph. In particular, each description segment center is\u00a0connected to its N = 3 nearest neighbors with respect to the geodesic distance of\u00a0the depth map to avoid connections across large depth changes. (c) Illustration\u00a0of the geometry term \u0001used for graph matching.<\/em><\/p>\n

\n

 <\/p>\n

Given ground truth matches, we can assign three class labels to each description segment matched by the graph matching algorithm:
\na) correct match; b) almost correct match and c) wrong match.\u00a0We compare our graph matching with conventional nearest neighbor (SIFT) matching and Hungarian matching of the description segments that admits at most one-to-one matching between description segments.<\/p>\n

 <\/p>\n

\n

\"Screenshot_072915_033041_PM\"<\/a><\/p>\n

Figure 02: Visual comparison of graph matching results. For illustration pur-\u00a0pose both RGB images are super-imposed. Green means a correct match (or\u00a0occlusion), blue is an almost correct match (de\fnition in text) and red is a\u00a0wrong match. Our result is clearly superior to the other techniques.<\/em><\/p>\n

\n

<\/h2>\n

Scene Flow<\/h2>\n

\"Screenshot_072915_033705_PM\"<\/a><\/p>\n

Figure 03 : results of the Scene Flow: (a) Original image pairs (b) using SphereFlow [Hornacek et at] (c) using SphereFlow initialized with\u00a0our Graph Matching (d) Our results using\u00a0GraphFlow.<\/p>\n