(Lecture) Distorted Space

Distorted Space (Ch. 6.1)

Consider a pixmap image with pixels \(1\times2\) mm. What is the distance from origo to the points \((0,10)\), \((10,0)\), and \((\sqrt{50},\sqrt{50})\approx(7,7)\)?
\(\psi: \mathbb{R}^3 \to \mathbb{R}^3\), \(\psi: \mathbb{X}\mapsto \mathbb{X}' = K\mathbb{X}\)
- recall that in camera calibration, we typically have \[K = \begin{bmatrix} s_x & s_\theta & o_x \\ 0 & s_y & o_y \\ 0 & 0 & 1 \end{bmatrix} \]
Redifining the Inner Product
- \(\langle\psi^{-1}(u),\psi^{-1}(v)\rangle = u^TK^{-T}K^{-1}v =\langle u,v\rangle_{K^{-T}K^{-1}} =\langle u,v\rangle_{S}\)
- where \(S=K^{-T}K^{-1}\)
Norm \(||u||_S=\sqrt{\langle u,u\rangle}\)
This gives rise to a distorted space
- angles are different
- norms are different

Movement in canonical space: \(X = RX_0+T\)
Co-ordinates in uncalibrated camera frame
- before: \(X_0' = KX_0\)
- after: \(X' = KX = KRX_0 + KT = KRK^{-1}X_0' + T'\)
- where \(T'=KT\)
Thus the movement in distorted (uncalibrated) space is \((R',T') = (KRK^{-1},KT)\)

The set of all Euclidean motions: \(\mathsf{SE}(3)=\{(R,T)|R\in\mathsf{SO}(3), T\in\mathbb{R}^3\}\)
Conjugate of \(\mathsf{SE}(3)\) \[G' = \bigg\{ g' = \begin{bmatrix} KRK^{-1} & T'\\0&1\end{bmatrix} \bigg|R\in\mathsf{SO}(3), T\in\mathbb{R}^3\bigg\}\]
Note commutative diagram in Fig 6.3 in the textbook

Calibrated (5.1) \(\lambda x = \Pi_0X\)
Uncalibrated (6.1) \(\lambda x' = K\Pi_0gX_0\)
- \(g\) is camera pose
- \(K\) is camera calibration matrix
- \(\Pi_0\) is the projection (as before)
Image transformation \(g: X_0 \mapsto X = KRX_0 + KT\)
- Uncalibrated: \(X' = KRK^{-1}X'_0+T'\)
- Projected:. \(\lambda x' = KRK^{-1}X'_0 + T'\) (homogeneous co-ordinates)
Rewriting in uncalibrated, heterogeneous co-ordinates:
- \(\lambda x'=KRK^{-1}X'_0 + T' = \Pi_0g'X_0'\)
Note \(\Pi_0\) translates from 3D/homogeneous to 2D.

Two views by the same camera. This gives one and the same calibration matrix \(K\) for both views.

Recall the calibrated case \[x_2^TEx_1 = 0\] where \(E=\hat TR\)
In the uncalibrated case, this becomes \[x_2'^TK^{-T}\hat TRK^{-1}x_1' = 0\] by substituting \(x=K^{-1}x'\)
We define the fundamental matrix \[F = K^{-T}\hat TRK^{-1} \quad\text{(eq. 6.10)}\]
This gives the epipolar constraint for uncalibrated cameras \[x_2^TFx_1 = 0 \quad\text{(eq. 6.8)}\]
This works essentially as in the calibrated case
In a perfect camera, \(K=I\) and \(F=E\)
It can be shown that \[F = \hat T' KRK^{-1} \quad\text{(eq. 6.14)}\] by invoking Lemma 5.4, but we’ll have to take this on trust.
\(F\) has rank two because \(\hat T'\) has rank two
- if \(F\) has full rank, find the SVD \(F=U\mathsf{diag}(\sigma_1,\sigma_2,\sigma_3)V^T\)
- replace \(F\) by \(U\mathsf{diag}(\sigma_1,\sigma_2,0)V^T\)
- more or less as in the calibrated case
Note that \(F\) has eight degrees of freedom
- \(\hat T'\) has two
- \(K\) has five
- \(R\) has three Hence it is impossible to recover \(\hat T'\) and \(R\) from \(F\), without additional information.
Many sources of additional information