# The Camera Projection

$\lambda x = K\Pi gX$

1. Euclidean transformation $$g$$ from world frame to camera frame
2. Projection $$\Pi$$ from 3D to 2D
3. Camera intrinsic transformation $$K$$

## Ambiguities

There are three ambiguities

$\lambda x = (KR^{-1})(R\Pi H^{-1}) (H gg_w^{-1})g_wX$

Due to the arbitrary choice of frame, $$R$$ and $$g_w$$ are inconsequential

# Equivalence classes of the calibration matrix

• $$K$$ can be chosen to be upper triangular without loss of generality

• decomposition $$K=QR$$ where
• $$R$$ is a rotation
• $$Q$$ is upper triangular
• The special linear group $$\mathsf{SL}(3)$$
• invertible matrices with determinant $$+1$$
• The group of rotations $$\mathsf{SO}(3) < \mathsf{SL}(3)$$ (subgroup)
• self-orthogonal $$R^{-1}=R^T$$
• Equivalence classes $\frac{\mathsf{SL}(3)}{\mathsf{SO}(3)}$
• equivalent because they induce the same iner product $$\langle\rangle_S$$
• $$S=K^{-T}K^{-1}$$
• $$(KR)^{-T}(KR)^{-1}=K^{-T}R^{-T}R^{-1}K^{-1}=K^{-T}RR^{-1}K^{-1}=K^{-T}K^{-1}=S$$
• one-to-finite correspondence between $$S$$ and upper triangular $$K$$
• usually only one $$K$$ is a valid camera calibration matrix
• We cannot distinguish $$K$$ from $$KR_0^T$$
• and $$g=[R,T]$$ from $$\tilde g=[R_0R,R_0T]$$

# Intrinsic and Extrinsic

• intrinsic $$K$$
• extrinsic $$g$$

$\Pi X = (\Pi H^{-1})(H X) = \tilde \Pi \tilde X$

• $$X$$ in the true world
• $$\tilde X$$ in the distorted world

# Stratified Reconstruction

• Projective $$\to$$ Affine $$\to$$ Euclidean

• Decompose $H^{-1}= \begin{bmatrix} K^{-1} & 0 \\ v^T & v_4 \end{bmatrix} = \begin{bmatrix} K^{-1} & 0 \\ 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} I & 0 \\ v^T & v_4 \end{bmatrix}$
• The first factor is an affine transformation
• The second factor is a projective transformation
• Equation 6.32 $F\mapsto ([I,0],[\hat\tilde T^TF,\tilde T])$