RepMatch: Robust Feature Matching and Pose
for Reconstructing Modern Cities

Introduction

Estimating the relative position of a pair of cameras has potential applications in diverse areas such as photometric estimation, system calibration and 3D reconstruction from images. However, such applications are seldom explored as two view pose estimation image pair has always been considered unstable. RepMatch represents our attempt to provide a framework for stable pose estimation. RepMatch is focused on handling the repetitive structures frequent in modern buildings, which represents the performance bottle-neck of pose estimation algorithms. This is achieved without any compromise of generality. At 10-20 seconds per image pair, RepMatch is slow. However, we consider it a proof of concept, that reliable pose estimation might one day be a reality. Paper, Supplementary, Matlab Code

Idea

PDF cannot be displayed.


RepMatch's framework is based on two concepts:

Firstly, even on scenes with strong repeated structures, it is possible to estimate an accurate-set of core matches. This is because correct matches tend to cluster at a specific motion, while wrong matches tend to be randomly distributed about a number of alternative positions. Thus, seeking out only the most consistent set of matches will give a highly stable core-set. Seen in (a).

Secondly, RepMatch introduces a means of reliably using the core-set of matches to verify alternative match hypotheses. The naive solution is to compute a core-set based epipolar geometry and use it for match verification. However, as mentioned earlier, such epipolar geometry estimation is unstable and vulnerable to quasi-degeneracies in the core-set a wrong epipolar constraint, would reject many true positives. Instead, we merge the core-set with clusters of match hypotheses to discover potential common geometry. This avoids true positive rejection at the expense of retaining some false positives. The remaining false positives are removed through another match consistency check.

By dividing the pose and correspondence problem into a series of robust steps, RepMatch brings overall stability to the pose estimation problem.

Results

PDF cannot be displayed.

RepMatch provides some crazily wide-baseline pose and correspondence estimates. Its not always stable at such extreme baselines. However, the results suggest high reliability at more conservative baseline settings.

PDF cannot be displayed.

Core-set recovered by matching a checker board to itself. Observe that the core-set is very robust even at extreme noise levels.

PDF cannot be displayed.

RepMatch can be integrated into a full Structure-from-Motion system like Visual SfM. This allows complete reconstructions in modern environments and prevents breakage when attempting to round corners. Warning: RepMatch is very slow when employed as an all-pair matching. We are working at fixing these speed issues.

Links:

BF: Bilateral Functions for Global Motion Modelling

This is the base paper for estimating match consistency and finding the core-set


CODE: Coherent decision boundaries for Feature Correspondence

This is a high speed implementation of BF used in RepMatch.


Visual SfM: A Visual Structure-from-Motion system

A GUI based Structure-from-Motion system that can be used for recovering 3D models from the correspondences obtained by RepMatch