---
title: Lecture Planar Scenes
categories: lecture
---

**Reading** Ma 2004:Ch 5

+ *Caveas* pages 122-124
+ *Planar Scenes* Section 5.3

# Review

Last week's exercise

1.  Not designed as an exercise to check that you have learnt what 
    I know.
    + Rather, it is designed as an experiment, as I would use it to
      test my own understanding.
2.  It sets up a closed loop.
    + the final result can be checked against the original data
3.  It demonstrates the eight-point algorithm, but it also demonstrates
    image capture (projection).
    - but this requires that you take the time to comprehend each step ...
4.  Suggested solution:
    [demo file](Python/eightpoint.ipynb) (Jupyter Notebook) does
    + thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the
core of 3D reconstruction ...

however, there is more

- the planar case
- uncalibrated cameras

# Degeneration

+ A plane $P$ is described by an equation
  $$N^T{X}=d$$
    + where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$
+ Consider object points $X_1,X_2,\ldots,X_n\in P$.
    + they all satisfy $N^TX_i = d$
    + or $\frac1dN^TX=1$ (1)
+ Extra constraint compared to the case for the eight-point algorithm
+ Consider the transformation between camera frames
  $$X'=RX+T$$
+ inserting from (1), we have
  $$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$
    + where $H=R+T\frac1dN^T$
    + $H$ depends on $(R,T)$ as well as $(N,d)$.
+ Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$.
    + we get $x'\sim Hx$ *(planar) homography*
+ multiplying both sides by $\hat x'$, we get
+ **Planar epipolar constraint**
  $$\hat x'Hx=0$$

Consider now why the eight-point algorithm fails

+ Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is
  orthogonal on $Hx$
+ Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus $\hat uH$ would be a valid essential matrix for any $u$
+ ... and the epipolar constraint is under-defined
+ it follows that the eight-point algorithm cannot work


# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$
so that
$$\forall i, \widehat{x_i'}^THx_i = 0$$

## Step 1.  First approximation of the homography matrix

1. Form the $\chi$ matrix as in the [Eight-point algorithm]().
2.  Compute the singular value decomposition of 
    $\chi=U_\chi\Sigma_\chi V_\chi^T$
3.  Let $H_L^s$ be the ninth column of $V_\chi$.
3.  Unstack $H_L^s$ to get $H_L$

*Note the similarity with the [Eight-point algorithm]().*


## Step 2.  Normalisation of the homography matrix

1. Let $\sigma_2$ be the second singular value of $H$ and normalise
   $$H=\frac{H_L}{\sigma_2}$$
2. Correct sign according to the depth constraint
   $${x'_i}^THx_i > 0 $$

## Step 3.  Decomposition of the homography matrix

1.  Decompose 
    $H^TH = V\Sigma V^T$
2.  Compute the four solutions for $(R,T/d,N)$.
    + The proof of Thm 5.19 is difficult to read
    + See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1)


| Parameter   | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 |
| :- | :- | :- | :-  | :- |
| $R_i$ |  $W_1U_1^T$ |  $W_2U_2^T$ |  $R_1$ | $R_2$ |
| $N_i$ |  $\hat v_2u_1$ |  $\hat v_2u_2$ | $-N_1$ | $-N_2$ |
| $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ |

where

+ $U_1=[ v_2, u_1, \hat v_2u_1 ]$
+ $U_2=[ v_2, u_2, \hat v_2u_2 ]$
+ $W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$
+ $W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$

where

+ $v_i$ are the three columns of $V$
+ $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3}
    {\sqrt{\sigma_1^2-\sigma_3^2}}$$
+ $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3}
    {\sqrt{\sigma_1^2-\sigma_3^2}}$$

# More theory

+ An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$
    + if $p\not\in P$, $x'$ only ends up on the epipolar line


# Homography versus Essential Matrix (5.3.4)

+ Piecewise planar scenes
    + Compute essential matrix from homographies
    + Compute both essential matrix and homographies from subsets
+ Theorem 5.21
    - $E=\hat TH$
    - $H^TE+E^TH = 0$
    - $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$