Revision b37f63fed33faf94344f5dbc52d4cf8dce83cdf1 (click the page title to view the current version)

Lecture Planar Scenes

Reading Ma 2004:Ch 5

  • Caveas pages 122-124
  • Planar Scenes Section 5.3

Review

Last week’s exercise

  1. Not designed as an exercise to check that you have learnt what I know.
    • Rather, it is designed as an experiment, as I would use it to test my own understanding.
  2. It sets up a closed loop.
    • the final result can be checked against the original data
  3. It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection).
    • but this requires that you take the time to comprehend each step …
  4. Suggested solution: demo file (Jupyter Notebook) does
    • thanks to Modestas for most of the programming

If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction …

however, there is more

  • the planar case
  • uncalibrated cameras

Degeneration

  • A plane \(P\) is described by an equation \[N^T{X}=d\]
    • where \(N=(n_1,n_2,n_3)\) is a vector orthogonal on \(P\)
  • Consider object points \(X_1,X_2,\ldots,X_n\in P\).
    • they all satisfy \(N^TX_i = d\)
    • or \(\frac1dN^TX=1\) (1)
  • Extra constraint compared to the case for the eight-point algorithm
  • Consider the transformation between camera frames \[X'=RX+T\]
  • inserting from (1), we have \[X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX\]
    • where \(H=R+T\frac1dN^T\)
    • \(H\) depends on \((R,T)\) as well as \((N,d)\).
  • Consider the image points \(x'=X'/\lambda'\) and \(x=X/\lambda\).
    • we get \(x'\sim Hx\) (planar) homography
  • multiplying both sides by \(\hat x'\), we get
  • Planar epipolar constraint \[\hat x'Hx=0\]

Consider now why the eight-point algorithm fails

  • Because \(x'\sim Hx\), for any \(u\in\mathbb{R}^3\), \(u\times x'=\hat ux'\) is orthogonal on \(Hx\)
  • Hence \(x'^T\hat u Hx=0\) for all \(u\in\mathbb{R}^3\)
  • thus \(\hat uH\) would be a valid essential matrix for any \(u\)
  • … and the epipolar constraint is under-defined
  • it follows that the eight-point algorithm cannot work

Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs \((x_i,x_i')\), this algorithm recovers \(H\) so that \[\forall i, \widehat{x_i'}^THx_i = 0\]

Step 1. First approximation of the homography matrix

  1. Form the \(\chi\) matrix as in the Eight-point algorithm.
  2. Compute the singular value decomposition of \(\chi=U_\chi\Sigma_\chi V_\chi^T\)
  3. Let \(H_L^s\) be the ninth column of \(V_\chi\).
  4. Unstack \(H_L^s\) to get \(H_L\)

Note the similarity with the Eight-point algorithm.

Step 2. Normalisation of the homography matrix

  1. Let \(\sigma_2\) be the second singular value of \(H\) and normalise \[H=\frac{H_L}{\sigma_2}\]
  2. Correct sign according to the depth constraint \[{x'_i}^THx_i > 0 \]

Step 3. Decomposition of the homography matrix

  1. Decompose \(H^TH = V\Sigma V^T\)
  2. Compute the four solutions for \((R,T/d,N)\).
Parameter Sol’n 1 Sol’n 2 Sol’n 3 Sol’n 4
\(R_i\) \(W_1U_1^T\) \(W_2U_2^T\) \(R_1\) \(R_2\)
\(N_i\) \(\hat v_2u_1\) \(\hat v_2u_2\) \(-N_1\) \(-N_2\)
\(T_i/d\) \((H-R_1)N_1\) \((H-R_2)N_2\) \(-T_1/d\) \(-T_2/d\)

where

  • \(U_1=[ v_2, u_1, \hat v_2u_1 ]\)
  • \(U_2=[ v_2, u_2, \hat v_2u_2 ]\)
  • \(W_1=[ Hv_2, Hu_1, \widehat{Hv_2}Hu_1 ]\)
  • \(W_2=[ Hv_2, Hu_2, \widehat{Hv_2}Hu_2 ]\)

where

  • \(v_i\) are the three columns of \(V\)
  • \[u_1 = \frac{\sqrt{1-\sigma_3^2}v_1+\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]
  • \[u_2 = \frac{\sqrt{1-\sigma_3^2}v_1-\sqrt{\sigma_1^2-1}v_3} {\sqrt{\sigma_1^2-\sigma_3^2}}\]

More theory

  • An image point \(x\) corresponding to \(p\in P\) uniquely determines \(x'\sim Hx\)
    • if \(p\not\in P\), \(x'\) only ends up on the epipolar line

Homography versus Essential Matrix (5.3.4)

  • Piecewise planar scenes
    • Compute essential matrix from homographies
    • Compute both essential matrix and homographies from subsets
  • Theorem 5.21
    • \(E=\hat TH\)
    • \(H^TE+E^TH = 0\)
    • \(H=\hat T^TE + Tv^T\) for some \(v\in\mathbb{R}\)