# Planar Lecture

## Changes from da482001415caa2c16fe181b600937501d0751c7 to b37f63fed33faf94344f5dbc52d4cf8dce83cdf1

---
title: Lecture Planar Scenes
categories: lecture
---

+ *Caveas* pages 122-124
+ *Planar Scenes* Section 5.3

# Review

Last week's exercise

1.  Not designed as an exercise to check that you have learnt what
I know.
+ Rather, it is designed as an experiment, as I would use it to
test my own understanding.
2.  It sets up a closed loop.
+ the final result can be checked against the original data
3.  It demonstrates the eight-point algorithm, but it also demonstrates
image capture (projection).
- but this requires that you take the time to comprehend each step ...
4.  Suggested solution:
[demo file](Python/eightpoint.ipynb) (Jupyter Notebook) does
+ thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the
core of 3D reconstruction ...

however, there is more

- the planar case
- uncalibrated cameras

# Degeneration

+ A plane $P$ is described by an equation
$$N^T{X}=d$$
+ where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$
+ Consider object points $X_1,X_2,\ldots,X_n\in P$.
+ they all satisfy $N^TX_i = d$
+ or $\frac1dN^TX=1$ (1)
+ Extra constraint compared to the case for the eight-point algorithm
+ Consider the transformation between camera frames
$$X'=RX+T$$
+ inserting from (1), we have
$$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$
+ where $H=R+T\frac1dN^T$
+ $H$ depends on $(R,T)$ as well as $(N,d)$.
+ Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$.
+ we get $x'\sim Hx$ *(planar) homography*
+ multiplying both sides by $\hat x'$, we get
+ **Planar epipolar constraint**
$$\hat x'Hx=0$$

Consider now why the eight-point algorithm fails

+ Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is
orthogonal on $Hx$
+ Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus $\hat uH$ would be a valid essential matrix for any $u$
+ ... and the epipolar constraint is under-defined
+ it follows that the eight-point algorithm cannot work

# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$
so that
$$\forall i, \widehat{x_i'}^THx_i = 0$$

## Step 1.  First approximation of the homography matrix

1. Form the $\chi$ matrix as in the [Eight-point algorithm]().
2.  Compute the singular value decomposition of
$\chi=U_\chi\Sigma_\chi V_\chi^T$
3.  Let $H_L^s$ be the ninth column of $V_\chi$.
3.  Unstack $H_L^s$ to get $H_L$

*Note the similarity with the [Eight-point algorithm]().*

## Step 2.  Normalisation of the homography matrix

1. Let $\sigma_2$ be the second singular value of $H$ and normalise
$$H=\frac{H_L}{\sigma_2}$$
2. Correct sign according to the depth constraint
$${x'_i}^THx_i > 0$$

## Step 3.  Decomposition of the homography matrix

1.  Decompose
$H^TH = V\Sigma V^T$
2.  Compute the four solutions for $(R,T/d,N)$.
+ The proof of Thm 5.19 is difficult to read
+ See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1)

| Parameter   | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 |
| :- | :- | :- | :-  | :- |
| $R_i$ |  $W_1U_1^T$ |  $W_2U_2^T$ |  $R_1$ | $R_2$ |
| $N_i$ |  $\hat v_2u_1$ |  $\hat v_2u_2$ | $-N_1$ | $-N_2$ |
| $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ |

where

+ $U_1=[ v_2, u_1, \hat v_2u_1 ]$
+ $U_2=[ v_2, u_2, \hat v_2u_2 ]$
+ $W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$
+ $W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$

where

+ $v_i$ are the three columns of $V$
+ $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$
+ $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$

# More theory

+ An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$
+ if $p\not\in P$, $x'$ only ends up on the epipolar line

# Homography versus Essential Matrix (5.3.4)

+ Piecewise planar scenes
+ Compute essential matrix from homographies
+ Compute both essential matrix and homographies from subsets
+ Theorem 5.21
- $E=\hat TH$
- $H^TE+E^TH = 0$
- $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$