# Distorted Space (Ch. 6.1)

## What is a distorted space?

1. Consider a pixmap image with pixels $$1\times2$$ mm. What is the distance from origo to the points $$(0,10)$$, $$(10,0)$$, and $$(\sqrt{50},\sqrt{50})\approx(7,7)$$?
2. $$\psi: \mathbb{R}^3 \to \mathbb{R}^3$$, $$\psi: \mathbb{X}\mapsto \mathbb{X}' = K\mathbb{X}$$
• recall that in camera calibration, we typically have $K = \begin{bmatrix} s_x & s_\theta & o_x \\ 0 & s_y & o_y \\ 0 & 0 & 1 \end{bmatrix}$
3. Redifining the Inner Product
• $$\langle\psi^{-1}(u),\psi^{-1}(v)\rangle = u^TK^{-T}K^{-1}v =\langle u,v\rangle_{K^{-T}K^{-1}} =\langle u,v\rangle_{S}$$
• where $$S=K^{-T}K^{-1}$$
4. Norm $$||u||_S=\sqrt{\langle u,u\rangle}$$
5. This gives rise to a distorted space
• angles are different
• norms are different

## 3D Motion in Distorted Space

1. Movement in canonical space: $$X = RX_0+T$$
2. Co-ordinates in uncalibrated camera frame
• before: $$X_0' = KX_0$$
• after: $$X' = KX = KRX_0 + KT = KRK^{-1}X_0' + T'$$
• where $$T'=KT$$
3. Thus the movement in distorted (uncalibrated) space is $$(R',T') = (KRK^{-1},KT)$$

## Conjugate Matrix Group

1. The set of all Euclidean motions: $$\mathsf{SE}(3)=\{(R,T)|R\in\mathsf{SO}(3), T\in\mathbb{R}^3\}$$
2. Conjugate of $$\mathsf{SE}(3)$$ $G' = \bigg\{ g' = \begin{bmatrix} KRK^{-1} & T'\\0&1\end{bmatrix} \bigg|R\in\mathsf{SO}(3), T\in\mathbb{R}^3\bigg\}$
3. Note commutative diagram in Fig 6.3 in the textbook

## Image Formation

1. Calibrated (5.1) $$\lambda x = \Pi_0X$$
2. Uncalibrated (6.1) $$\lambda x' = K\Pi_0gX_0$$
• $$g$$ is camera pose
• $$K$$ is camera calibration matrix
• $$\Pi_0$$ is the projection (as before)
3. Image transformation $$g: X_0 \mapsto X = KRX_0 + KT$$
• Uncalibrated: $$X' = KRK^{-1}X'_0+T'$$
• Projected:. $$\lambda x' = KRK^{-1}X'_0 + T'$$ (homogeneous co-ordinates)
4. Rewriting in uncalibrated, heterogeneous co-ordinates:
• $$\lambda x'=KRK^{-1}X'_0 + T' = \Pi_0g'X_0'$$
5. Note $$\Pi_0$$ translates from 3D/homogeneous to 2D.

# Uncalibrated Epipolar Geometry (Ch. 6.2)

Two views by the same camera. This gives one and the same calibration matrix $$K$$ for both views.

• Recall the calibrated case $x_2^TEx_1 = 0$ where $$E=\hat TR$$
• In the uncalibrated case, this becomes $x_2'^TK^{-T}\hat TRK^{-1}x_1' = 0$ by substituting $$x=K^{-1}x'$$
• We define the fundamental matrix $F = K^{-T}\hat TRK^{-1} \quad\text{(eq. 6.10)}$
• This gives the epipolar constraint for uncalibrated cameras $x_2^TFx_1 = 0 \quad\text{(eq. 6.8)}$
• This works essentially as in the calibrated case
• In a perfect camera, $$K=I$$ and $$F=E$$
• It can be shown that $F = \hat T' KRK^{-1} \quad\text{(eq. 6.14)}$ by invoking Lemma 5.4, but we’ll have to take this on trust.
• $$F$$ has rank two because $$\hat T'$$ has rank two
• if $$F$$ has full rank, find the SVD $$F=U\mathsf{diag}(\sigma_1,\sigma_2,\sigma_3)V^T$$
• replace $$F$$ by $$U\mathsf{diag}(\sigma_1,\sigma_2,0)V^T$$
• more or less as in the calibrated case
• Note that $$F$$ has eight degrees of freedom
• $$\hat T'$$ has two
• $$K$$ has five
• $$R$$ has three Hence it is impossible to recover $$\hat T'$$ and $$R$$ from $$F$$, without additional information.
• Many sources of additional information