# Lecture Planar Scenes

**Reading** Ma 2004:Ch 5

*Caveats*pages 122-124*Planar Scenes*Section 5.3

# Review

Last week’s exercise

- Not designed as an exercise to check that you have learnt what I know.
- Rather, it is designed as an experiment, as I would use it to test my own understanding.

- It sets up a closed loop.
- the final result can be checked against the original data

- It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection).
- but this requires that you take the time to comprehend each step …

- Suggested solution: demo file (Jupyter Notebook) does
- thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction …

however, there is more

- the planar case
- uncalibrated cameras

# Degeneration

- A plane \(P\) is described by an equation \[N^T{X}=d\]
- where \(N=(n_1,n_2,n_3)\) is a vector orthogonal on \(P\)

- Consider object points \(X_1,X_2,\ldots,X_n\in P\).
- they all satisfy \(N^TX_i = d\)
- or \(\frac1dN^TX=1\) (1)

- Extra constraint compared to the case for the eight-point algorithm
- Consider the transformation between camera frames \[X'=RX+T\]
- inserting from (1), we have \[X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX\]
- where \(H=R+T\frac1dN^T\)
- \(H\) depends on \((R,T)\) as well as \((N,d)\).

- Consider the image points \(x'=X'/\lambda'\) and \(x=X/\lambda\).
- we get \(x'\sim Hx\)
*(planar) homography*

- we get \(x'\sim Hx\)
- multiplying both sides by \(\hat x'\), we get
**Planar epipolar constraint**\[\hat x'Hx=0\]

Consider now why the eight-point algorithm fails

- Because \(x'\sim Hx\), for any \(u\in\mathbb{R}^3\), \(u\times x'=\hat ux'\) is orthogonal on \(Hx\)
- Hence \(x'^T\hat u Hx=0\) for all \(u\in\mathbb{R}^3\)
- thus \(\hat uH\) would be a valid essential matrix for any \(u\)
- … and the epipolar constraint is under-defined
- it follows that the eight-point algorithm cannot work

# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs \((x_i,x_i')\), this algorithm recovers \(H\) so that \[\forall i, \widehat{x_i'}^THx_i = 0\]

## Step 1. First approximation of the homography matrix

- Form the \(\chi\) matrix as in the Eight-point algorithm.
- Compute the singular value decomposition of \(\chi=U_\chi\Sigma_\chi V_\chi^T\)
- Let \(H_L^s\) be the ninth column of \(V_\chi\).
- Unstack \(H_L^s\) to get \(H_L\)

*Note the similarity with the Eight-point algorithm.*

## Step 2. Normalisation of the homography matrix

- Let \(\sigma_2\) be the second singular value of \(H\) and normalise \[H=\frac{H_L}{\sigma_2}\]
- Correct sign according to the depth constraint \[{x'_i}^THx_i > 0 \]

## Step 3. Decomposition of the homography matrix

- Decompose \(H^TH = V\Sigma V^T\)
- Compute the four solutions for \((R,T/d,N)\).
- The proof of Thm 5.19 is difficult to read
- See a more complete discussion

Parameter | Sol’n 1 | Sol’n 2 | Sol’n 3 | Sol’n 4 |
---|---|---|---|---|

\(R_i\) | \(W_1U_1^T\) | \(W_2U_2^T\) | \(R_1\) | \(R_2\) |

\(N_i\) | \(\hat v_2u_1\) | \(\hat v_2u_2\) | \(-N_1\) | \(-N_2\) |

\(T_i/d\) | \((H-R_1)N_1\) | \((H-R_2)N_2\) | \(-T_1/d\) | \(-T_2/d\) |

where

- \(U_1=[ v_2, u_1, \hat v_2u_1 ]\)
- \(U_2=[ v_2, u_2, \hat v_2u_2 ]\)
- \(W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]\)
- \(W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]\)

where

- \(v_i\) are the three columns of \(V\)
- \[u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]
- \[u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]

# More theory

- An image point \(x\) corresponding to \(p\in P\) uniquely determines \(x'\sim Hx\)
- if \(p\not\in P\), \(x'\) only ends up on the epipolar line

# Homography versus Essential Matrix (5.3.4)

- Piecewise planar scenes
- Compute essential matrix from homographies
- Compute both essential matrix and homographies from subsets

- Theorem 5.21
- \(E=\hat TH\)
- \(H^TE+E^TH = 0\)
- \(H=\hat T^TE + Tv^T\) for some \(v\in\mathbb{R}\)