Reading Ma 2004:Ch 5

• Caveas pages 122-124
• Planar Scenes Section 5.3

Review

Last week’s exercise

1. Not designed as an exercise to check that you have learnt what I know.
   • Rather, it is designed as an experiment, as I would use it to test my own understanding.
2. It sets up a closed loop.
   • the final result can be checked against the original data
3. It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection).
   • but this requires that you take the time to comprehend each step …
4. Suggested solution: demo file (Jupyter Notebook) does
   • thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction …

however, there is more

• the planar case
• uncalibrated cameras

Degeneration

• A plane \(P\) is described by an equation \[N^T{X}=d\]
  • where \(N=(n_1,n_2,n_3)\) is a vector orthogonal on \(P\)
• Consider object points \(X_1,X_2,\ldots,X_n\in P\).
  • they all satisfy \(N^TX_i = d\)
  • or \(\frac1dN^TX=1\) (1)
• Extra constraint compared to the case for the eight-point algorithm
• Consider the transformation between camera frames \[X'=RX+T\]
• inserting from (1), we have \[X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX\]
  • where \(H=R+T\frac1dN^T\)
  • \(H\) depends on \((R,T)\) as well as \((N,d)\).
• Consider the image points \(x'=X'/\lambda'\) and \(x=X/\lambda\).
  • we get \(x'\sim Hx\) (planar) homography
• multiplying both sides by \(\hat x'\), we get
• Planar epipolar constraint \[\hat x'Hx=0\]

Consider now why the eight-point algorithm fails

• Because \(x'\sim Hx\), for any \(u\in\mathbb{R}^3\), \(u\times x'=\hat ux'\) is orthogonal on \(Hx\)
• Hence \(x'^T\hat u Hx=0\) for all \(u\in\mathbb{R}^3\)
• thus \(\hat uH\) would be a valid essential matrix for any \(u\)
• … and the epipolar constraint is under-defined
• it follows that the eight-point algorithm cannot work

Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs \((x_i,x_i')\), this algorithm recovers \(H\) so that \[\forall i, \widehat{x_i'}^THx_i = 0\]

Step 1. First approximation of the homography matrix

1. Form the \(\chi\) matrix as in the Eight-point algorithm.
2. Compute the singular value decomposition of \(\chi=U_\chi\Sigma_\chi V_\chi^T\)
3. Let \(H_L^s\) be the ninth column of \(V_\chi\).
4. Unstack \(H_L^s\) to get \(H_L\)

Note the similarity with the Eight-point algorithm.

Step 2. Normalisation of the homography matrix

1. Let \(\sigma_2\) be the second singular value of \(H\) and normalise \[H=\frac{H_L}{\sigma_2}\]
2. Correct sign according to the depth constraint \[{x'_i}^THx_i > 0 \]

Step 3. Decomposition of the homography matrix

1. Decompose \(H^TH = V\Sigma V^T\)
2. Compute the four solutions for \((R,T/d,N)\).
   • The proof of Thm 5.19 is difficult to read
   • See a more complete discussion

where

• \(U_1=[ v_2, u_1, \hat v_2u_1 ]\)
• \(U_2=[ v_2, u_2, \hat v_2u_2 ]\)
• \(W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]\)
• \(W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]\)

where

• \(v_i\) are the three columns of \(V\)
• \[u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]
• \[u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]

More theory

• An image point \(x\) corresponding to \(p\in P\) uniquely determines \(x'\sim Hx\)
  • if \(p\not\in P\), \(x'\) only ends up on the epipolar line

Homography versus Essential Matrix (5.3.4)

• Piecewise planar scenes
  • Compute essential matrix from homographies
  • Compute both essential matrix and homographies from subsets
• Theorem 5.21
  • \(E=\hat TH\)
  • \(H^TE+E^TH = 0\)
  • \(H=\hat T^TE + Tv^T\) for some \(v\in\mathbb{R}\)