Lecture - Stratified Reconstruction
The Camera Projection
\[\lambda x = K\Pi gX\]
- Euclidean transformation \(g\) from world frame to camera frame
- Projection \(\Pi\) from 3D to 2D
- Camera intrinsic transformation \(K\)
Ambiguities
There are three ambiguities
\[\lambda x = (KR^{-1})(R\Pi H^{-1}) (H gg_w^{-1})g_wX\]
Due to the arbitrary choice of frame, \(R\) and \(g_w\) are inconsequential
Equivalence classes of the calibration matrix
\(K\) can be chosen to be upper triangular without loss of generality
- decomposition \(K=QR\) where
- \(R\) is a rotation
- \(Q\) is upper triangular
- The special linear group \(\mathsf{SL}(3)\)
- invertible matrices with determinant \(+1\)
- The group of rotations \(\mathsf{SO}(3) < \mathsf{SL}(3)\) (subgroup)
- self-orthogonal \(R^{-1}=R^T\)
- Equivalence classes \[\frac{\mathsf{SL}(3)}{\mathsf{SO}(3)}\]
- equivalent because they induce the same iner product \(\langle\rangle_S\)
- \(S=K^{-T}K^{-1}\)
- \((KR)^{-T}(KR)^{-1}=K^{-T}R^{-T}R^{-1}K^{-1}=K^{-T}RR^{-1}K^{-1}=K^{-T}K^{-1}=S\)
- one-to-finite correspondence between \(S\) and upper triangular \(K\)
- usually only one \(K\) is a valid camera calibration matrix
- We cannot distinguish \(K\) from \(KR_0^T\)
- and \(g=[R,T]\) from \(\tilde g=[R_0R,R_0T]\)
Intrinsic and Extrinsic
- intrinsic \(K\)
- extrinsic \(g\)
\[\Pi X = (\Pi H^{-1})(H X) = \tilde \Pi \tilde X\]
- \(X\) in the true world
- \(\tilde X\) in the distorted world
Stratified Reconstruction
Projective \(\to\) Affine \(\to\) Euclidean
- Decompose \[H^{-1}=
\begin{bmatrix}
K^{-1} & 0 \\ v^T & v_4
\end{bmatrix}
=
\begin{bmatrix}
K^{-1} & 0 \\ 0 & 1
\end{bmatrix}
\cdot
\begin{bmatrix}
I & 0 \\ v^T & v_4
\end{bmatrix}
\]
- The first factor is an affine transformation
- The second factor is a projective transformation
Equation 6.32 \[F\mapsto ([I,0],[\hat\tilde T^TF,\tilde T])\]
