Revision a4b2c99b7d0706d475a0cd2598e6a5dca3c96247 (click the page title to view the current version)
Changes from a4b2c99b7d0706d475a0cd2598e6a5dca3c96247 to current
---
title: Distorted Space
categories: session
---
**Briefing** [Distorted Lecture]()
**Additional Reading**
Chapter 9 in *OpenCV 3 Computer Vision with Python Cookbook* by
Alexey Spizhevoy (author).
Search for it in [Oria](https://oria.no/).
There is an e-book available.
# Exercises
This exercise builds on the
[Eight-point algorithm]() but does not assume calibrated cameras.
## An Example of a Distorted space
## Step 1. Make a Data Set
### Step 1. Preliminaries.
1. Take two images of the same scene, using different camera poses.
- the difference between the poses should be significant,
but small enough to recognise the same feature points.
- i.e. two consecutive frames from a video will probably be too similar.
2. Run the Harris Detector on both images, and identify at least
eight features which you can pair between the images.
- if you do not find eight, you need to use more similar poses.
1. Consider a triangle in the object space, formed by the three
vertices, $A=(0,0,10)$, $B=(20,0,10)$, and $C=(20,10,10)$.
Let
+ $\alpha$ be the angle between the vectors $\overrightarrow{AB}$
and $\overrightarrow{AC}$.
+ $\beta$ be the angle between the vectors $\overrightarrow{BA}$
and $\overrightarrow{BC}$.
**Draw** a figure to show this information.
2. **Calculate** the angles $\alpha$ and $\beta$.
3. Consider an ideal perspective camera, posed such that the camera
frame matches the world frame.
**Calculate** the image points corresponding to $A$, $B$, and $C$.
Assume that the image plane has $z=1$.
**Draw** an image to display this projection.
4. **Calculate** the angles $\alpha'$ and $\beta'$
in the image, corresponding to $\alpha$ and $\beta$.
Do they match the original angles?
5. **Calculate** the lengths of the edges of the triangles in the image.
6. Make a new drawing of the image plane,
displaying the quantities calculated in 4 and 5.
Keep it for later reference.
**Note 1**
you should pair the feature points manually in this exercise, to make
sure that no mismatches ruin your results.
When you have the first prototype working, you can try to pair feature
points programmatically, using SIFT or other methods to match features.
Recall the formula for the angle $\theta$ between two vectors
$u$ and $v$
$$\theta = \cos^{-1}\frac{u\cdot v}{||u||\cdot||v||}.$$
This is found in any elementary calculus book.
You did not need this above, because the triangles had an orthogonal
angle, but you will need it below.
## Step 2. Fundamental matrix
### Step 2. Distortion.
Each pair $(\mathbf{x}_i,\mathbf{x}_i')$ of corresponding features
from the two images give rise to a Kronecker product.
$$\mathbf{a}_i=\mathbf{x}_i\otimes\mathbf{x}_i'$$
Using these vectors as columns, we get the $8\times n$ matrix $\chi$,
where $n$ is the number of feature pairs.
Suppose the camera is not ideal, but instead has calibration matrix
$$K =
\begin{bmatrix}
2 & \frac1{\sqrt{3}} & 4 \\ 0 & 1 & 4 \\ 0 & 0 & 1
\end{bmatrix}$$
1. Calculate $\chi$ from your dataset.
2. Compute the singular value decomposition of
$\chi=U_\chi\Sigma_\chi V_\chi^T$
using `numpy.linalg.svd`
- Inspect the three components. What do you see?
- The middle component should be a diagonal matrix containing the
singular values. How are they ordered?
4. According to Ma (2004:121) the serialised fundamental matrix $E^s$
is the ninth column of $V_\chi$.
- this should be wrig
5. Unserialise $E^s$ to get the fundamental matrix $E$.
- you can do this with the numpy `reshape` function, but test it.
You may have to transpose the resulting matrix.
## Step 3. Projection onto the Essential Space
7. **Calculate** the image points corresponding to $A$, $B$, and $C$,
using the camera calibration matrix $K$.
(See Ma 2004:55)
8. **Draw** the resulting image to scale.
9. **Calculate** the lengths of the edges of the image of the triangle.
For now, we use the familiar formulæ for vector norms.
10. Let $\alpha'$ and $\beta'$ be the images of the angles $\alpha$
and $\beta$.
**Calculate** these to image angles and add the information to you figure.
11. Compare your distorted image to the original image from Step 1.
If $E=U\Sigma V^T$ is the fundamental matrix, we get the essential
matrix by replacing $\Sigma$ with $\mathsf{diag}(1,1,0)$
(cf. Ma (2004:121)).
### Step 3. The Distorted Inner Product Space
## Step 4. Rotation and Translation
Recall the textbook's definition of an inner product in distorted space
$$\langle u,v\rangle_S = u^TSv \quad\text{where } S = K^{-T}K^{-1},$$
for some `camera calibration' matrix $K$.
We are going to redo Step 2, using this distorted inner product.
Recover Translation and Rotation matrices from the essential matrix,
relying on Ma (2004:121).
12. [9] Calculate the edges (norms) of the triangle, using the
distorted inner product to define the norm.
13. [10] Calculate the angles $\alpha'$ and $\beta'$, using
the distorted inner product and the formula
$$\theta = \cos^{-1}\frac{u\cdot v}{||u||\cdot||v||}.$$
14. Make a figure illustrating the quantities calculated.
15. [11] Compare the edges and angles thus calculated with the
calculations from Step 2 and from Step 1.
Reflect on your observations.
What do the two solutions represent?
### Step 4. Reflection
Does it seem reasonable compared to how you moved the cameras?
Think through the following:
16. What does this exercise tell you about camera distortion and
inner product spaces?
17. Is there anything you do not quite understand?
Is there something missing in the exercise, something that should
have been tested?
## From the Textbook
+ Exercise 6.4
# Debrief
+ [Step 1](Solutions/distorted-step1.pdf)
+ [Step 2(7-10)](Solutions/distorted-step2.pdf)
+ Step 2(11)
+ Note how the $x$-axis is stretched, doubling the length of $AB$.
+ Not also how it is skewed, bending the angle $\beta'$ from
$90°$ to $120°$.
+ Step 3(11)
+ [Sample python code](Python/distorted.py)
+ Inverting matrices by hand is usually not time well invested,
hence the python code. It is possible though, and for some
it may be quicker.
