Reading Ma 2004:Ch 5

Review

Last weeks exercise

1. Not designed as an exercise to check that you have learnt what I know.
   • Rather, it is designed as an experiment, as I would use it to test my own understanding.
2. It sets up a closed loop.
   • the final result can be checked against the original data
3. It demonstrates the eight-point algorithm, but it also demonstrates image capture (projection).
   • but this requires that you take the time to comprehend each step …

If you can complete and comprehend all the steps, you have understood the core of 3D reconstruction …

however, there is more

• the planar case
• uncalibrated cameras

Degeneration

• A plane \(P\) is described by an equation \[N^T{X}=d\]
  • where \(N=(n_1,n_2,n_3)\) is a vector orthogonal on \(P\)
• Consider object points \(X_1,X_2,\ldots,X_n\in P\).
  • they all satisfy \(N^TX_i = d\)
  • or \(\frac1dN^TX=1\) (1)
• Extra constraint compared to the case for the eight-point algorithm
• Consider the transformation between camera frames \[X'=RX+T\]
• inserting from (1), we have \[X'=RX+T\frac1dN^TX=(R+T\frac1dN^T)X=HX\]
  • where \(H=R+T\frac1dN^T\)
  • \(H\) depends on \((R,T)\) as well as \((N,d)\).
• Consider the image points \(x'=X'/\lambda'\) and \(x=X/\lambda\).
  • we get \(x'\sim Hx\) (planar) homography
• multiplying both sides by \(\hat x'\), we get
• Planar epipolar constraint \[\hat x'Hx=0\]
• because \(x'\sim Hx\), for any \(u\in\mathbb{R}^3\), \(u\times x'=\hat ux'\bot Hx\)
• hence \(x'^T\hat u Hx=0\) for all \(u\in\mathbb{R}^3\)
• thus the epipolar constraint is under-defined
• it follows that the eight-point algorithm cannot work

Four-Point Algorithm for Planar Scenes (Alg 5.2 page 139)

Given at least four image pairs \((x_i,x_i')\), this algorithm recovers \(H\) so that \[\forall i, \bighat{x_i'}^THx_i = 0\]

Step 1. First approximation of the homography matrix

1. Form the \(\chi\) matrix as in the Eight-point algorithm.
2. Compute the singular value decomposition of \(\chi=U_\chi\Sigma_\chi V_\chi^T\) 3 Let \(H_L^s\) be the ninth column of \(V_\chi\). 3 Unstack \(H_L^s\) to get \(H_L\)

Note the similarity with the Eight-point algorithm.

Step 2. Normalisation of the homography matrix

1. Let \(\sigma_2\) be the second singular value of \(H\) and normalise \[H=\frac{H_L}{\sigma_2}\]
2. Correct sign TODO

Step 3. Decomposition of the homography matrix

1. Decompose \(H^TH = V\Sigma V^T\)$
2. Compute the four solutions for \((R,T/d,N)\). See Thm 5.19 TODO

More theory

• An image point \(x\) corresponding to \(p\in P\) uniquely determines \(x'\sim Hx\)
  • if \(p\not\in P\), \(x'\) only ends up on the epipolar line

Homography versus Essential Matrix (5.3.4)