Egomotion Estimation with Large Field-of-View Vision

John Lim's Research Page

Contact email:
john {dot} lim {at} rsise.anu.edu.au or
Enable images to see address

This page summarizes my research done on Egomotion Estimation with Large Field-of-View Vision as a PhD student under the supervision of Dr. Nick Barnes at NICTA and the Australian National University between the years 2006 and 2009.

** Thesis Supplementary Videos here **

Publications List
Research Summary
Links

Publications List
In reverse chronological order. See also Research Summary for a brief description of key publications.

Egomotion Estimation with Large Field-of-View Vision
J. Lim
PhD Thesis, Australian National University, 2009.
videos

Convex Relaxation of Mixture Regression with Efficient Algorithms
N. Quadrianto, T. Caetano, J. Lim and D. Schuurmans
Advances in Neural Information Processing Systems (NIPS), 2009.
[PDF]

Estimating Relative Camera Motion from the Antipodal-Epipolar Constraint
J. Lim, N. Barnes and H. Li
Pattern Analysis and Machine Intelligence (PAMI) under review, 2009.

Robust Visual Homing with Landmark Angles
J. Lim and N. Barnes
Proceedings of Robotics Science and Systems (RSS 09), Seattle, 2009.
[PDF] videos

Estimation of the Epipole using Optical Flow at Antipodal Points
J. Lim and N. Barnes
Computer Vision and Image Understanding (CVIU), accepted, 2009.
[PDF preprint] videos publisher's version

Real Time Motion Recovery using a Hemispherical Sensor
R.G. Dengate, N. Barnes, J. Lim, C. Luu and R. Guymer
Proceedings of the Australasian Conference on Robotics and Automation (ACRA), 2008.

Directions of Egomotion from Antipodal Points
J. Lim and N. Barnes
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) oral paper, 2008.
[PDF] videos (static scene) videos (non-static scene)

Estimation of the Epipole using Optical Flow at Antipodal Points
J. Lim and N. Barnes
Proceedings of IEEE ICCV Omnidirectional Vision Workshop (OMNIVIS), 2007.
[PDF] videos Matlab code

Insect Inspired Robotics
J. Lim, C. McCarthy, D. Shaw, N. Barnes and L.Cole
Proceedings of The Australasian Conference on Robotics and Automation (ACRA), Auckland, 2006.
[PDF]

Research Summary

Here, I summarize some of the papers I have published with a paragraph or two each, and some explanatory diagrams.
My work investigated the problem of egomotion estimation in a monocular, large Field-of-View (FOV) camera from two views of the scene. I developed new constraints and algorithms that exploited the larger information content of wide FOV images and the geometry of image spheres in order to aid and simplify the egomotion recovery task.

Inspiration

The inspiration for this work is illustrated in below. A narrow FOV observer (the picture of a human) undergoing a sideways translation may observe an optical flow field that looks somewhat similar to that observed if it were to instead rotate about some vertical axis. Even though the two motions are very different, estimating them from the observed optical flow is difficult.

Conversely, consider the large FOV observer below (picture of a bird), where the two optic flow fields resulting from a similar translation and rotation are vastly different. Under a forward translation, both the bird's eyes see optical flow directed backwards; whereas under a rotation, the optic flow in the two eyes are directed in opposite directions.

Antipodal Points

The above observation is well known among vision researchers and is often cited to illustrate the advantages of large FOV vision. Here, we investigate this phenomenon mathematically, and derive a number of novel constraints on egomotion from this simple idea.
We consider a geometrical feature - the antipodal point pair - which is found only in central cameras where the FOV is in excess of a hemisphere (the diagram below shows some examples). Two rays are antipodal if the rays are collinear but point in opposite directions (see diagram below) - that is, for a calibrated camera with the origin at the projection centre, r and -r are antipodal. If world points R₁ and R₂ lay on these rays, the image motion of these points as the camera moves will give rise to constraints on the camera translation and rotation.

Directions of Flow at Antipodal Points [CVPR 08]

Consider a pair of antipodal points r_1

and

, where the optical flow is measured to be $\dot{r}_1$ and $\dot{r}_2$ . We showed that purely from the directions of these flow vectors (and without using their magnitudes), one may constrain the direction of translation and the axis of rotation to some subset of the image sphere. In fact, the direction of translation must lie within a lune (the name in geometry for these wedge-shaped regions) $A_{res}(tr)$ shown as the shaded region in the left diagram below. $A_{res}(tr)$ is bounded by two great circles $C_{bound,1}$ and $C_{bound,2}$ , such that $\dot{r}_1$ lies in the plane of circle $C_{bound,1}$ and $\dot{r}_2$ lies in the plane of circle $C_{bound,2}$ . Likewise, the rotation axis must lie within the lune (shaded region $A_{res}(rot)$ ) bounded by two other great circles $C_{bound,1}$ and $C_{bound,2}$ where $\dot{r}_1$ is now perpendicular to the plane of circle $C_{bound,1}$ , and $\dot{r}_2$ is also perpendicular to the plane of the circle $C_{bound,2}$ .

From the flow at many such pairs of antipodal points, a set of regions constraining the direction of translation will arise. The translation direction must lie in the intersection of all these regions. Simultaneously, a different set of regions constraining the direction of the rotation axis may be obtained, and their intersection will give an estimate of the rotation axis. One possible algorithm for egomotion recovery is a Hough-reminescent approach that votes on each of these regions in 2D, with the peak in the vote space giving a robust (to outliers) solution. For further details, refer to our CVPR08 paper.

Translation from the Flow at Antipodal Points [CVIU 09] [OMNIVIS 07]

We also showed that by summing the optic flow at antipodal points, one may constrain the direction of translation to lie on a plane. This is illustrated below, where $\dot(r}_s = \dot(r}_1 + \dot(r}_2$ , and either

will give the normal vector of a plane in which translation

is constrained to lie. (The intersection of that plane and the image sphere gives the great circle C.) Furthermore, two or more such antipodal flow pairs will then give multiple such planes (or great circles) and translation may be found from taking the intersection of these planes. Once translation has been found, rotation may be recovered also via a second step.

A number of algorithms using these constraints are possible, and our CVIU paper gives two algorithms for robust estimation of motion. These are a RANSAC-type algorithm as well as a Hough-voting type algorithm. The algorithms performed competitvely with the state-of-the art, and in particular, the voting method works in constant time under increasing outlier proportions.

Visual Homing - Relaxed Egomotion [RSS 09]

Whilst highly accurate estimates of egomotion are nice, the computational requirements of obtaining them may be too high for certain platforms - eg. a light, flying robot with a payload of a few hundred grams, and hence quite limited computing resources. One observes that egomotion estimates that are not very accurate or precise are still useful for many applications, as long as certain bounds on the maximum error of the estimate are met. Such estimates may be much faster and computationally less demanding to recover, since the requirements for accuracy have been relaxed.
Here, we study this problem in the context of the visual homing task, where a mobile agent or robot sees a view of the world at its current location and compares that with a remembererd view of how the world looked like from its home location. From the two views, the robot attempts to figure out some sensible directions to move in, that will take it back to the home location. In our work, the angle between any two landmark points in the scene gives rise to a constraint on the translation between the current and home locations. From many such constraints, we derive estimates of translation that are inaccurate but have measurable bounds on the maximum estimate error. As long as this maximum error meets certain easily satisfiable conditions, we show that the robot will provably converge on the home location. The algorithms provided are based on a voting framework and are robust to outliers and noise.
Depicted below (from left to right) is the robot used for experiments, the view from a large FOV fish-eye lens camera mounted on the robot, and an example of a panoramic image used for homing experiments.

Links

My supervisor, Nick Barnes' home page.
Chris Mccarthy, who does research in approximately similar areas.
The Insectbot page.

counters