Revision f6988610783d9bb283e65b39a819ce18f20118c4 (click the page title to view the current version)
Eight-point algorithm
Briefing Eight-point algorithm Lecture
Additional Reading Chapter 9 in OpenCV 3 Computer Vision with Python Cookbook by Alexey Spizhevoy (author). Search for it in Oria. There is an e-book available.
Exercises
In this exercise, we shall try to determine the relative pose of two cameras, using the eight-point algorithm (or a variant thereof).
Step 1. Make a Data Set
- Take two images of the same scene, using different camera poses.
- the difference between the poses should be significant, but small enough to recognise the same feature points.
- i.e. two consecutive frames from a video will probably be too similar.
- Run the Harris Detector on both images, and identify at least eight features which you can pair between the images.
- if you do not find eight, you need to use more similar poses.
Note 1 It may be useful to calibrate the camera(s) and undistort the images before starting. It is ok to try without calibration first, for the sake of simplicity.
Note 2 you should pair the feature points manually in this exercise, to make sure that no mismatches ruin your results. When you have the first prototype working, you can try to pair feature points programmatically, using SIFT or other methods to match features.
Step 2. Fundamental matrix
Each pair \((\mathbf{x}_i,\mathbf{x}_i')\) of corresponding features from the two images give rise to a Kronecker product. \[\mathbf{a}_i=\mathbf{x}_i\otimes\mathbf{x}_i'\] Using these vectors as columns, we get the \(8\times n\) matrix \(\chi\), where \(n\) is the number of feature pairs.
- Calculate \(\chi\) from your dataset.
- Compute the singular value decomposition of \(\chi=U_\chi\Sigma_\chi V_\chi^T\) using
numpy.linalg.svd
- Inspect the three components. What do you see?
- The middle component should be a diagonal matrix containing the singular values. How are they ordered?
- According to Ma (2004:121) the serialised fundamental matrix \(E^s\) is the ninth column of \(V_\chi\).
- this should be wrig
- Unserialise \(E^s\) to get the fundamental matrix \(E\).
- you can do this with the numpy
reshape
function, but test it. You may have to transpose the resulting matrix.
- you can do this with the numpy
Step 3. Projection onto the Essential Space
If \(E=U\Sigma V^T\) is the fundamental matrix, we get the essential matrix by replacing \(\Sigma\) with \(\mathsf{diag}(1,1,0)\) (cf. Ma (2004:121)).
Step 4. Rotation and Translation
Recover Translation and Rotation matrices from the essential matrix, relying on Ma (2004:121).
What do the two solutions represent?
Does it seem reasonable compared to how you moved the cameras?