## Revision b37f63fed33faf94344f5dbc52d4cf8dce83cdf1 (click the page title to view the current version)

# Planar Lecture

## Changes from b37f63fed33faf94344f5dbc52d4cf8dce83cdf1 to b24b00a279239219204da68e9b3ccdd48fda6933

--- title: Lecture Planar Scenes categories: lecture --- **Reading** Ma 2004:Ch 5 + *Caveas* pages 122-124 + *Caveats* pages 122-124 + *Planar Scenes* Section 5.3 # Review Last week's exercise 1. Not designed as an exercise to check that you have learnt what I know. + Rather, it is designed as an experiment, as I would use it to test my own understanding. 2. It sets up a closed loop. + the final result can be checked against the original data 3. It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection). - but this requires that you take the time to comprehend each step ... 4. Suggested solution: [demo file](Python/eightpoint.ipynb) (Jupyter Notebook) does + thanks to Modestas for most of the programming If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction ... however, there is more - the planar case - uncalibrated cameras # Degeneration + A plane $P$ is described by an equation $$N^T{X}=d$$ + where $N=(n_1,n_2,n_3)$ is a vector orthogonal on $P$ + Consider object points $X_1,X_2,\ldots,X_n\in P$. + they all satisfy $N^TX_i = d$ + or $\frac1dN^TX=1$ (1) + Extra constraint compared to the case for the eight-point algorithm + Consider the transformation between camera frames $$X'=RX+T$$ + inserting from (1), we have $$X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX$$ + where $H=R+T\frac1dN^T$ + $H$ depends on $(R,T)$ as well as $(N,d)$. + Consider the image points $x'=X'/\lambda'$ and $x=X/\lambda$. + we get $x'\sim Hx$ *(planar) homography* + multiplying both sides by $\hat x'$, we get + **Planar epipolar constraint** $$\hat x'Hx=0$$ Consider now why the eight-point algorithm fails + Because $x'\sim Hx$, for any $u\in\mathbb{R}^3$, $u\times x'=\hat ux'$ is orthogonal on $Hx$ + Hence $x'^T\hat u Hx=0$ for all $u\in\mathbb{R}^3$ + thus $\hat uH$ would be a valid essential matrix for any $u$ + ... and the epipolar constraint is under-defined + it follows that the eight-point algorithm cannot work # Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139) Given at least four image pairs $(x_i,x_i')$, this algorithm recovers $H$ so that $$\forall i, \widehat{x_i'}^THx_i = 0$$ ## Step 1. First approximation of the homography matrix 1. Form the $\chi$ matrix as in the [Eight-point algorithm](). 2. Compute the singular value decomposition of $\chi=U_\chi\Sigma_\chi V_\chi^T$ 3. Let $H_L^s$ be the ninth column of $V_\chi$. 3. Unstack $H_L^s$ to get $H_L$ *Note the similarity with the [Eight-point algorithm]().* ## Step 2. Normalisation of the homography matrix 1. Let $\sigma_2$ be the second singular value of $H$ and normalise $$H=\frac{H_L}{\sigma_2}$$ 2. Correct sign according to the depth constraint $${x'_i}^THx_i > 0 $$ ## Step 3. Decomposition of the homography matrix 1. Decompose $H^TH = V\Sigma V^T$ 2. Compute the four solutions for $(R,T/d,N)$. + The proof of Thm 5.19 is difficult to read + See [a more complete discussion](https://hal.archives-ouvertes.fr/inria-00174036v1) | Parameter | Sol'n 1 | Sol'n 2 | Sol'n 3 | Sol'n 4 | | :- | :- | :- | :- | :- | | $R_i$ | $W_1U_1^T$ | $W_2U_2^T$ | $R_1$ | $R_2$ | | $N_i$ | $\hat v_2u_1$ | $\hat v_2u_2$ | $-N_1$ | $-N_2$ | | $T_i/d$ | $(H-R_1)N_1$ | $(H-R_2)N_2$ | $-T_1/d$ | $-T_2/d$ | where + $U_1=[ v_2, u_1, \hat v_2u_1 ]$ + $U_2=[ v_2, u_2, \hat v_2u_2 ]$ + $W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]$ + $W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]$ where + $v_i$ are the three columns of $V$ + $$u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$ + $$u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}$$ # More theory + An image point $x$ corresponding to $p\in P$ uniquely determines $x'\sim Hx$ + if $p\not\in P$, $x'$ only ends up on the epipolar line # Homography versus Essential Matrix (5.3.4) + Piecewise planar scenes + Compute essential matrix from homographies + Compute both essential matrix and homographies from subsets + Theorem 5.21 - $E=\hat TH$ - $H^TE+E^TH = 0$ - $H=\hat T^TE + Tv^T$ for some $v\in\mathbb{R}$