# Planar Lecture

## Changes from 68f91952d84e01e140483f2b087592cefa25cc57 to da482001415caa2c16fe181b600937501d0751c7

---
title: Lecture Planar Scenes
categories: lecture
---

# Review

Last week's exercise

1.  Not designed as an exercise to check that you have learnt what
I know.
+ Rather, it is designed as an experiment, as I would use it to
test my own understanding.
2.  It sets up a closed loop.
+ the final result can be checked against the original data
3.  It demonstrates the eight-point algorithm, but it also demonstrates
image capture (projection).
- but this requires that you take the time to comprehend each step ...

If you can complete and comprehend all the steps, you have understood the
core of 3D reconstruction ...

however, there is more

- the planar case
- uncalibrated cameras

# Degeneration

+ A plane $P$ is described by an equation
$$N^T{X}=d$$
+ where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$
+ Consider object points $X_1,X_2,\ldots,X_n\in P$.
+ they all satisfy $N^TX_i = d$
+ or $\frac1dN^TX=1$ (1)
+ Extra constraint compared to the case for the eight-point algorithm
+ Consider the transformation between camera frames
$$X'=RX+T$$
+ inserting from (1), we have
$$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$
+ where $H=R+T\frac1dN^T$
+ $H$ depends on $(R,T)$ as well as $(N,d)$.
+ Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$.
+ we get $x'\sim Hx$ *(planar) homography*
+ multiplying both sides by $\hat x'$, we get
+ **Planar epipolar constraint**
$$\hat x'Hx=0$$

Consider now why the eight-point algorithm fails

+ Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is
orthogonal on $Hx$
+ Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus $\hat uH$ would be a valid essential matrix for any $u$
+ ... and the epipolar constraint is under-defined
+ it follows that the eight-point algorithm cannot work

# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$
so that
$$\forall i, \widehat{x_i'}^THx_i = 0$$

## Step 1.  First approximation of the homography matrix

1. Form the $\chi$ matrix as in the [Eight-point algorithm]().
2.  Compute the singular value decomposition of
$\chi=U_\chi\Sigma_\chi V_\chi^T$
3  Let $H_L^s$ be the ninth column of $V_\chi$.
3  Unstack $H_L^s$ to get $H_L$
3.  Let $H_L^s$ be the ninth column of $V_\chi$.
3.  Unstack $H_L^s$ to get $H_L$

*Note the similarity with the [Eight-point algorithm]().*

## Step 2.  Normalisation of the homography matrix

1. Let $\sigma_2$ be the second singular value of $H$ and normalise
$$H=\frac{H_L}{\sigma_2}$$
2. Correct sign according to the depth constraint
$${x'_i}^THx_i > 0$$

## Step 3.  Decomposition of the homography matrix

1.  Decompose
$H^TH = V\Sigma V^T$
2.  Compute the four solutions for $(R,T/d,N)$.
+ The proof of Thm 5.19 is difficult to read
+ See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1)

| Parameter   | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 |
| :- | :- | :- | :-  | :- |
| $R_i$ |  $W_1U_1^T$ |  $W_2U_2^T$ |  $R_1$ | $R_2$ |
| $N_i$ |  $\hat v_2u_1$ |  $\hat v_2u_2$ | $-N_1$ | $-N_2$ |
| $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ |

where

+ $U_1=[ v_2, u_1, \hat v_2u_1 ]$
+ $U_2=[ v_2, u_2, \hat v_2u_2 ]$
+ $W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$
+ $W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$

where

+ $v_i$ are the three columns of $V$
+ $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$
+ $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$

# More theory

+ An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$
+ if $p\not\in P$, $x'$ only ends up on the epipolar line

# Homography versus Essential Matrix (5.3.4)

+ Piecewise planar scenes
+ Compute essential matrix from homographies
+ Compute both essential matrix and homographies from subsets
+ Theorem 5.21
- $E=\hat TH$
- $H^TE+E^TH = 0$
- $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$