# Lecture Planar Scenes

• Caveats pages 122-124
• Planar Scenes Section 5.3

# Review

Last week’s exercise

1. Not designed as an exercise to check that you have learnt what I know.
• Rather, it is designed as an experiment, as I would use it to test my own understanding.
2. It sets up a closed loop.
• the final result can be checked against the original data
3. It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection).
• but this requires that you take the time to comprehend each step …
4. Suggested solution: demo file (Jupyter Notebook) does
• thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction …

however, there is more

• the planar case
• uncalibrated cameras

# Degeneration

• A plane $$P$$ is described by an equation $N^T{X}=d$
• where $$N=(n_1,n_2,n_3)$$ is a vector orthogonal on $$P$$
• Consider object points $$X_1,X_2,\ldots,X_n\in P$$.
• they all satisfy $$N^TX_i = d$$
• or $$\frac1dN^TX=1$$ (1)
• Extra constraint compared to the case for the eight-point algorithm
• Consider the transformation between camera frames $X'=RX+T$
• inserting from (1), we have $X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$
• where $$H=R+T\frac1dN^T$$
• $$H$$ depends on $$(R,T)$$ as well as $$(N,d)$$.
• Consider the image points $$x'=X'/\lambda'$$ and $$x=X/\lambda$$.
• we get $$x'\sim Hx$$ (planar) homography
• multiplying both sides by $$\hat x'$$, we get
• Planar epipolar constraint $\hat x'Hx=0$

Consider now why the eight-point algorithm fails

• Because $$x'\sim Hx$$, for any $$u\in\mathbb{R}^3$$, $$u\times x'=\hat ux'$$ is orthogonal on $$Hx$$
• Hence $$x'^T\hat u Hx=0$$ for all $$u\in\mathbb{R}^3$$
• thus $$\hat uH$$ would be a valid essential matrix for any $$u$$
• … and the epipolar constraint is under-defined
• it follows that the eight-point algorithm cannot work

# Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs $$(x_i,x_i')$$, this algorithm recovers $$H$$ so that $\forall i, \widehat{x_i'}^THx_i = 0$

## Step 1. First approximation of the homography matrix

1. Form the $$\chi$$ matrix as in the Eight-point algorithm.
2. Compute the singular value decomposition of $$\chi=U_\chi\Sigma_\chi V_\chi^T$$
3. Let $$H_L^s$$ be the ninth column of $$V_\chi$$.
4. Unstack $$H_L^s$$ to get $$H_L$$

Note the similarity with the Eight-point algorithm.

## Step 2. Normalisation of the homography matrix

1. Let $$\sigma_2$$ be the second singular value of $$H$$ and normalise $H=\frac{H_L}{\sigma_2}$
2. Correct sign according to the depth constraint ${x'_i}^THx_i > 0$

## Step 3. Decomposition of the homography matrix

1. Decompose $$H^TH = V\Sigma V^T$$
2. Compute the four solutions for $$(R,T/d,N)$$.
Parameter Sol’n 1 Sol’n 2 Sol’n 3 Sol’n 4
$$R_i$$ $$W_1U_1^T$$ $$W_2U_2^T$$ $$R_1$$ $$R_2$$
$$N_i$$ $$\hat v_2u_1$$ $$\hat v_2u_2$$ $$-N_1$$ $$-N_2$$
$$T_i/d$$ $$(H-R_1)N_1$$ $$(H-R_2)N_2$$ $$-T_1/d$$ $$-T_2/d$$

where

• $$U_1=[ v_2, u_1, \hat v_2u_1 ]$$
• $$U_2=[ v_2, u_2, \hat v_2u_2 ]$$
• $$W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$$
• $$W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$$

where

• $$v_i$$ are the three columns of $$V$$
• $u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$
• $u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$

# More theory

• An image point $$x$$ corresponding to $$p\in P$$ uniquely determines $$x'\sim Hx$$
• if $$p\not\in P$$, $$x'$$ only ends up on the epipolar line

# Homography versus Essential Matrix (5.3.4)

• Piecewise planar scenes
• Compute essential matrix from homographies
• Compute both essential matrix and homographies from subsets
• Theorem 5.21
• $$E=\hat TH$$
• $$H^TE+E^TH = 0$$
• $$H=\hat T^TE + Tv^T$$ for some $$v\in\mathbb{R}$$