Revision bccd4a766a109d01688c301695f44f3545efdc8f (click the page title to view the current version)

Planar Lecture

Changes from bccd4a766a109d01688c301695f44f3545efdc8f to 3174e0ba0ccefe48c6011a52898eb0d0da8c1480

---
title: Lecture Planar Scenes
categories: lecture
---

**Reading** Ma 2004:Ch 5

# Review

Last week's exercise

1.  Not designed as an exercise to check that you have learnt what 
    I know.
    + Rather, it is designed as an experiment, as I would use it to
      test my own understanding.
2.  It sets up a closed loop.
    + the final result can be checked against the original data
3.  It demonstrates the eight-point algorithm, but it also demonstrates
    image capture (projection).
    - but this requires that you take the time to comprehend each step ...

If you can complete and comprehend all the steps, you have understood the
core of 3D reconstruction ...

however, there is more

- the planar case
- uncalibrated cameras

# Degeneration

+ A plane $P$ is described by an equation
  $$N^T{X}=d$$
    + where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$
+ Consider object points $X_1,X_2,\ldots,X_n\in P$.
    + they all satisfy $N^TX_i = d$
    + or $\frac1dN^TX=1$ (1)
+ Extra constraint compared to the case for the eight-point algorithm
+ Consider the transformation between camera frames
  $$X'=RX+T$$
+ inserting from (1), we have
  $$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$
    + where $H=R+T\frac1dN^T$
    + $H$ depends on $(R,T)$ as well as $(N,d)$.
+ Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$.
    + we get $x'\sim Hx$ *(planar) homography*
+ multiplying both sides by $\hat x'$, we get
+ **Planar epipolar constraint**
  $$\hat x'Hx=0$$
+ because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'\bot Hx$
+ hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus the epipolar constraint is under-defined

Consider now why the eight-point algorithm fails

+ Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is
  orthogonal on $Hx$
+ Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$
+ thus $\hat uH$ would be a valid essential matrix for any $u$
+ ... and the epipolar constraint is under-defined
+ it follows that the eight-point algorithm cannot work


# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$
so that
$$\forall i, \widehat{x_i'}^THx_i = 0$$

## Step 1.  First approximation of the homography matrix

1. Form the $\chi$ matrix as in the [Eight-point algorithm]().
2.  Compute the singular value decomposition of 
    $\chi=U_\chi\Sigma_\chi V_\chi^T$
3  Let $H_L^s$ be the ninth column of $V_\chi$.
3  Unstack $H_L^s$ to get $H_L$

*Note the similarity with the [Eight-point algorithm]().*


## Step 2.  Normalisation of the homography matrix

1. Let $\sigma_2$ be the second singular value of $H$ and normalise
   $$H=\frac{H_L}{\sigma_2}$$
2. Correct sign according to the depth constraint
   $${x'_i}^THx_i > 0 $$

## Step 3.  Decomposition of the homography matrix

1.  Decompose 
    $H^TH = V\Sigma V^T$$
2.  Compute the four solutions for $(R,T/d,N)$.
    + The proof of Thm 5.19 is difficult to read
    + See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1)


| Parameter   | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 |
| :- | :- | :- | :-  | :- |
| $R_i$ |  $W_1U_1^T$ |  $W_2U_2^T$ |  $R_1$ | $R_2$ |
| $N_i$ |  $\hat v_2u_1$ |  $\hat v_2u_2$ | $-N_1$ | $-N_2$ |
| $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ |

where

+ $U_1=[ v_2, u_1, \hat v_2u_1 ]$
+ $U_2=[ v_2, u_2, \hat v_2u_2 ]$
+ $W_1=[ Hv_2, Hu_1, \widehat{H\hat v_2}Hu_1 ]$
+ $W_2=[ Hv_2, Hu_2, \widehat{H\hat v_2}Hu_2 ]$

where

+ $v_i$ are the three columns of $V$
+ $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3}
    {\sqrt{\sigma_1^2-\sigma_3^2}}$$
+ $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3}
    {\sqrt{\sigma_1^2-\sigma_3^2}}$$

# More theory

+ An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$
    + if $p\not\in P$, $x'$ only ends up on the epipolar line


# Homography versus Essential Matrix (5.3.4)

+ Piecewise planar scenes
    + Compute essential matrix from homographies
    + Compute both essential matrix and homographies from subsets
+ Theorem 5.21
    - $E=\hat TH$
    - $H^TE+E^TH = 0$
    - $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$