Revision afd164f56fc119bb46f60b9239207cd258d0e268 (click the page title to view the current version)

Distorted Space

Changes from afd164f56fc119bb46f60b9239207cd258d0e268 to a4b2c99b7d0706d475a0cd2598e6a5dca3c96247

---
title: Distorted Space
categories: session
---

**Briefing** [Distorted Lecture]()

# Exercise
**Additional Reading**
Chapter 9 in  *OpenCV 3 Computer Vision with Python Cookbook* by
Alexey Spizhevoy (author).
Search for it in [Oria](https://oria.no/).
There is an e-book available.

# Exercises

This exercise builds on the
[Eight-point algorithm]() but does not assume calibrated cameras.

## Step 1.  Make a Data Set

1.  Take two images of the same scene, using different camera poses.
    - the difference between the poses should be significant,
      but small enough to recognise the same feature points.
    - i.e. two consecutive frames from a video will probably be too similar.
2.  Run the Harris Detector on both images, and identify at least
    eight features which you can pair between the images.
    - if you do not find eight, you need to use more similar poses.

**Note 1**
you should pair the feature points manually in this exercise, to make
sure that no mismatches ruin your results. 
When you have the first prototype working, you can try to pair feature
points programmatically, using SIFT or other methods to match features.

## Step 2.  Fundamental matrix

Each pair $(\mathbf{x}_i,\mathbf{x}_i')$ of corresponding features
from the two images give rise to a Kronecker product.
$$\mathbf{a}_i=\mathbf{x}_i\otimes\mathbf{x}_i'$$
Using these vectors as columns, we get the $8\times n$ matrix $\chi$,
where $n$ is the number of feature pairs.

1.  Calculate $\chi$ from your dataset.
2.  Compute the singular value decomposition of 
    $\chi=U_\chi\Sigma_\chi V_\chi^T$
    using `numpy.linalg.svd`
    - Inspect the three components.  What do you see?
    - The middle component should be a diagonal matrix containing the
      singular values.  How are they ordered?
4.  According to Ma (2004:121) the serialised fundamental matrix $E^s$
    is the ninth column of $V_\chi$.
    - this should be wrig
5.  Unserialise $E^s$ to get the fundamental matrix $E$.
    - you can do this with the numpy `reshape` function, but test it.
      You may have to transpose the resulting matrix.
  
## Step 3.  Projection onto the Essential Space

If $E=U\Sigma V^T$ is the fundamental matrix, we get the essential
matrix by replacing $\Sigma$ with $\mathsf{diag}(1,1,0)$
(cf. Ma (2004:121)).

## Step 4.  Rotation and Translation

Recover Translation and Rotation matrices from the essential matrix,
relying on Ma (2004:121).

What do the two solutions represent?

Does it seem reasonable compared to how you moved the cameras?

# Debrief