3D Reconstruction (continuing Eight-point algorithm)

Briefing Eight-point algorithm Lecture

Additional Reading Chapter 9 in OpenCV 3 Computer Vision with Python Cookbook by Alexey Spizhevoy (author). Search for it in Oria. There is an e-book available.

Learning Outcome See how the eight-point algorithm can be fitted into a complete system to do 3D reconstruction from a real stereo view.

Exercises

In this exercise, we shall try to determine the relative pose of two cameras, using the eight-point algorithm (or a variant thereof).

Step 1. Make a Data Set

  1. Take two images of the same scene, using different camera poses.
    • the difference between the poses should be significant, but small enough to recognise the same feature points.
    • i.e. two consecutive frames from a video will probably be too similar.
  2. Use SIFT (or similar) to find and match at least eight feature pairs in the two images
    • if you do not find eight, you need to use more similar poses.

Note 1 It may be useful to calibrate the camera(s) and undistort the images before starting. It is ok to try without calibration first, for the sake of simplicity.

Note 2 you should validate the feature matching visually first, to make sure that an error here does not spoil the continuation.

Step 2. Eight-Point Algorithm

Use the Eight-point algorithm from the previous exercise (Part 2) to recover the relative pose \((R,T)\) between the cameras.

Does the transformation \((R,T)\) seem reasonable?
What does it mean in terms of rotation and translation in the real world space?

Step 3. 3D Reconstruction

Calculate 3D co-ordinates in the global frame for each of the features from Step 1. Do the co-ordinates seem reasonable?

Step 4. (Optional) Visualisation

Visualise the reconstructed points in 3D, using for instance matplotlib in python.

Step 5. (Optional) Automatic Matching

Extend your system to use SIFT to automatically match features in Step 1.

Debrief

This example is under construction, and there is a problem with the recovery of the relative pose. It does, however, exemplify the broad structure of a solution;