{"id":4630,"date":"2019-07-29T09:40:51","date_gmt":"2019-07-29T09:40:51","guid":{"rendered":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/?page_id=4630"},"modified":"2020-05-04T08:37:30","modified_gmt":"2020-05-04T08:37:30","slug":"neural-guided-ransac","status":"publish","type":"page","link":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/","title":{"rendered":"Neural-Guided RANSAC"},"content":{"rendered":"<h3>Abstract:<\/h3>\n<p>We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled fashion that lets us optimize an arbitrary task loss during training, leading to large improvements on classic computer vision tasks. We present two further extensions to NG-RANSAC. Firstly, using the inlier count itself as training signal allows us to train neural guidance in a self-supervised fashion. Secondly, we combine neural guidance with differentiable RANSAC to build neural networks which focus on certain parts of the input data and make the output predictions as good as possible. We evaluate NG-RANSAC on a wide array of computer vision tasks, namely estimation of epipolar geometry, horizon line estimation and camera re-localization. We achieve superior or competitive results compared to state-of-the-art robust estimators, including very recent, learned ones.<\/p>\n<h3>Poster:<\/h3>\n<p><a href=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/iccv19_ngransac_poster_small.pdf\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-4759\" src=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-1024x498.png\" alt=\"\" width=\"720\" height=\"350\" srcset=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-1024x498.png 1024w, https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-300x146.png 300w, https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-768x374.png 768w, https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster.png 1767w\" sizes=\"auto, (max-width: 720px) 100vw, 720px\" \/><\/a><\/p>\n<h3>Results:<\/h3>\n<p>Estimation of epipolar geometry:<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/ehaZ7FhDNS0\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>Estimation of horizon lines:<\/p>\n<p><a href=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/vid.gif\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4825\" src=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/vid.gif\" alt=\"\" width=\"256\" height=\"256\" \/><\/a>Camera Re-Localization:<\/p>\n<h3><a href=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/output.gif\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4901\" src=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/output.gif\" alt=\"\" width=\"256\" height=\"176\" \/><\/a><\/h3>\n<h3>Code:<\/h3>\n<p>NG-RANSAC for estimation of essential or fundamental matrices:\u00a0<a href=\"https:\/\/github.com\/vislearn\/ngransac\/\">https:\/\/github.com\/vislearn\/ngransac\/<\/a><br \/>\nNG-DSAC for horizon line estimation:\u00a0<a href=\"https:\/\/github.com\/vislearn\/ngdsac_horizon\">https:\/\/github.com\/vislearn\/ngdsac_horizon<\/a><br \/>\nNG-DSAC for camera re-localizaton: <a href=\"https:\/\/github.com\/vislearn\/ngdsac_camreloc\">https:\/\/github.com\/vislearn\/ngdsac_camreloc<\/a><\/p>\n<p><strong>Note:<\/strong> NG-RANSAC has also been extended to CONSAC (CVPR 2020) for multi-model fitting (<a href=\"https:\/\/arxiv.org\/abs\/2001.02643\">paper<\/a>, <a href=\"https:\/\/github.com\/fkluger\/consac\">code<\/a>)<\/p>\n<h3>Publication:<\/h3>\n<p>E. Brachmann and C. Rother, &#8220;Neural-Guided RANSAC: Learning Where to Sample Model Hypothesis&#8221;, ICCV 2019. [<a href=\"https:\/\/arxiv.org\/pdf\/1905.04132.pdf\">pdf<\/a>]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Abstract: We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":0,"parent":125,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"inline_featured_image":false,"footnotes":""},"class_list":["post-4630","page","type-page","status-publish","hentry","post"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg\" \/>\n<meta property=\"og:description\" content=\"Abstract: We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/\" \/>\n<meta property=\"og:site_name\" content=\"Computer Vision and Learning Lab Heidelberg\" \/>\n<meta property=\"article:modified_time\" content=\"2020-05-04T08:37:30+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-1024x498.png\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/\",\"url\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/\",\"name\":\"Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/ngransac_poster-1024x498.png\",\"datePublished\":\"2019-07-29T09:40:51+00:00\",\"dateModified\":\"2020-05-04T08:37:30+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/#primaryimage\",\"url\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/ngransac_poster.png\",\"contentUrl\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/ngransac_poster.png\",\"width\":1767,\"height\":860},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/neural-guided-ransac\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research\",\"item\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/research\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Neural-Guided RANSAC\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/#website\",\"url\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/\",\"name\":\"Computer Vision and Learning Lab Heidelberg\",\"description\":\"Computer Vision and Learning Lab Heidelberg\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/hci.iwr.uni-heidelberg.de\\\/vislearn\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/","og_locale":"en_US","og_type":"article","og_title":"Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg","og_description":"Abstract: We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled [&hellip;]","og_url":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/","og_site_name":"Computer Vision and Learning Lab Heidelberg","article_modified_time":"2020-05-04T08:37:30+00:00","og_image":[{"url":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-1024x498.png","type":"","width":"","height":""}],"twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/","url":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/","name":"Neural-Guided RANSAC - Computer Vision and Learning Lab Heidelberg","isPartOf":{"@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/#website"},"primaryImageOfPage":{"@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/#primaryimage"},"image":{"@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/#primaryimage"},"thumbnailUrl":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster-1024x498.png","datePublished":"2019-07-29T09:40:51+00:00","dateModified":"2020-05-04T08:37:30+00:00","breadcrumb":{"@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/#primaryimage","url":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster.png","contentUrl":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-content\/uploads\/2019\/07\/ngransac_poster.png","width":1767,"height":860},{"@type":"BreadcrumbList","@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/neural-guided-ransac\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/"},{"@type":"ListItem","position":2,"name":"Research","item":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/research\/"},{"@type":"ListItem","position":3,"name":"Neural-Guided RANSAC"}]},{"@type":"WebSite","@id":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/#website","url":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/","name":"Computer Vision and Learning Lab Heidelberg","description":"Computer Vision and Learning Lab Heidelberg","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/pages\/4630","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/comments?post=4630"}],"version-history":[{"count":11,"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/pages\/4630\/revisions"}],"predecessor-version":[{"id":4903,"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/pages\/4630\/revisions\/4903"}],"up":[{"embeddable":true,"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/pages\/125"}],"wp:attachment":[{"href":"https:\/\/hci.iwr.uni-heidelberg.de\/vislearn\/wp-json\/wp\/v2\/media?parent=4630"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}