Lecture - Stratified Reconstruction

The Camera Projection

\[\lambda x = K\Pi gX\]

  1. Euclidean transformation \(g\) from world frame to camera frame
  2. Projection \(\Pi\) from 3D to 2D
  3. Camera intrinsic transformation \(K\)


There are three ambiguities

\[\lambda x = (KR^{-1})(R\Pi H^{-1}) (H gg_w^{-1})g_wX\]

Due to the arbitrary choice of frame, \(R\) and \(g_w\) are inconsequential

Equivalence classes of the calibration matrix

  • \(K\) can be chosen to be upper triangular without loss of generality

  • decomposition \(K=QR\) where
    • \(R\) is a rotation
    • \(Q\) is upper triangular
  • The special linear group \(\mathsf{SL}(3)\)
    • invertible matrices with determinant \(+1\)
  • The group of rotations \(\mathsf{SO}(3) < \mathsf{SL}(3)\) (subgroup)
    • self-orthogonal \(R^{-1}=R^T\)
  • Equivalence classes \[\frac{\mathsf{SL}(3)}{\mathsf{SO}(3)}\]
    • equivalent because they induce the same iner product \(\langle\rangle_S\)
    • \(S=K^{-T}K^{-1}\)
    • \((KR)^{-T}(KR)^{-1}=K^{-T}R^{-T}R^{-1}K^{-1}=K^{-T}RR^{-1}K^{-1}=K^{-T}K^{-1}=S\)
    • one-to-finite correspondence between \(S\) and upper triangular \(K\)
    • usually only one \(K\) is a valid camera calibration matrix
  • We cannot distinguish \(K\) from \(KR_0^T\)
    • and \(g=[R,T]\) from \(\tilde g=[R_0R,R_0T]\)

Intrinsic and Extrinsic

  • intrinsic \(K\)
  • extrinsic \(g\)

\[\Pi X = (\Pi H^{-1})(H X) = \tilde \Pi \tilde X\]

  • \(X\) in the true world
  • \(\tilde X\) in the distorted world

Stratified Reconstruction

  • Projective \(\to\) Affine \(\to\) Euclidean

  • Decompose \[H^{-1}= \begin{bmatrix} K^{-1} & 0 \\ v^T & v_4 \end{bmatrix} = \begin{bmatrix} K^{-1} & 0 \\ 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} I & 0 \\ v^T & v_4 \end{bmatrix} \]
    • The first factor is an affine transformation
    • The second factor is a projective transformation
  • Equation 6.32 \[F\mapsto ([I,0],[\hat\tilde T^TF,\tilde T])\]