Revision b37f63fed33faf94344f5dbc52d4cf8dce83cdf1 (click the page title to view the current version)
Changes from b37f63fed33faf94344f5dbc52d4cf8dce83cdf1 to b24b00a279239219204da68e9b3ccdd48fda6933
---
title: Lecture Planar Scenes
categories: lecture
---
**Reading** Ma 2004:Ch 5
+ *Caveas* pages 122-124
+ *Caveats* pages 122-124
+ *Planar Scenes* Section 5.3
# Review
Last week's exercise
1. Not designed as an exercise to check that you have learnt what
I know.
+ Rather, it is designed as an experiment, as I would use it to
test my own understanding.
2. It sets up a closed loop.
+ the final result can be checked against the original data
3. It demonstrates the eight-point algorithm, but it also demonstrates
image capture (projection).
- but this requires that you take the time to comprehend each step ...
4. Suggested solution:
[demo file](Python/eightpoint.ipynb) (Jupyter Notebook) does
+ thanks to Modestas for most of the programming
If you can complete and comprehend all the steps, you have understood the
core of 3D reconstruction ...
however, there is more
- the planar case
- uncalibrated cameras
# Degeneration
+ A plane $P$ is described by an equation
$$N^T{X}=d$$
+ where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$
+ Consider object points $X_1,X_2,\ldots,X_n\in P$.
+ they all satisfy $N^TX_i = d$
+ or $\frac1dN^TX=1$ (1)
+ Extra constraint compared to the case for the eight-point algorithm
+ Consider the transformation between camera frames
$$X'=RX+T$$
+ inserting from (1), we have
$$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$
+ where $H=R+T\frac1dN^T$
+ $H$ depends on $(R,T)$ as well as $(N,d)$.
+ Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$.
+ we get $x'\sim Hx$ *(planar) homography*
+ multiplying both sides by $\hat x'$, we get
+ **Planar epipolar constraint**
$$\hat x'Hx=0$$
Consider now why the eight-point algorithm fails
+ Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is
orthogonal on $Hx$
+ Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus $\hat uH$ would be a valid essential matrix for any $u$
+ ... and the epipolar constraint is under-defined
+ it follows that the eight-point algorithm cannot work
# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)
Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$
so that
$$\forall i, \widehat{x_i'}^THx_i = 0$$
## Step 1. First approximation of the homography matrix
1. Form the $\chi$ matrix as in the [Eight-point algorithm]().
2. Compute the singular value decomposition of
$\chi=U_\chi\Sigma_\chi V_\chi^T$
3. Let $H_L^s$ be the ninth column of $V_\chi$.
3. Unstack $H_L^s$ to get $H_L$
*Note the similarity with the [Eight-point algorithm]().*
## Step 2. Normalisation of the homography matrix
1. Let $\sigma_2$ be the second singular value of $H$ and normalise
$$H=\frac{H_L}{\sigma_2}$$
2. Correct sign according to the depth constraint
$${x'_i}^THx_i > 0 $$
## Step 3. Decomposition of the homography matrix
1. Decompose
$H^TH = V\Sigma V^T$
2. Compute the four solutions for $(R,T/d,N)$.
+ The proof of Thm 5.19 is difficult to read
+ See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1)
| Parameter | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 |
| :- | :- | :- | :- | :- |
| $R_i$ | $W_1U_1^T$ | $W_2U_2^T$ | $R_1$ | $R_2$ |
| $N_i$ | $\hat v_2u_1$ | $\hat v_2u_2$ | $-N_1$ | $-N_2$ |
| $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ |
where
+ $U_1=[ v_2, u_1, \hat v_2u_1 ]$
+ $U_2=[ v_2, u_2, \hat v_2u_2 ]$
+ $W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$
+ $W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$
where
+ $v_i$ are the three columns of $V$
+ $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3}
{\sqrt{\sigma_1^2-\sigma_3^2}}$$
+ $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3}
{\sqrt{\sigma_1^2-\sigma_3^2}}$$
# More theory
+ An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$
+ if $p\not\in P$, $x'$ only ends up on the epipolar line
# Homography versus Essential Matrix (5.3.4)
+ Piecewise planar scenes
+ Compute essential matrix from homographies
+ Compute both essential matrix and homographies from subsets
+ Theorem 5.21
- $E=\hat TH$
- $H^TE+E^TH = 0$
- $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$