Camera Calibration

Linear Calibration and Intrinsic Camera Properties

Hans Georg Schaathun

NTNU, Noregs Teknisk-Naturvitskaplege Universitet

September 2023

Let's look again at our camera system, drawn here in 3D perspective. Please, count the number of co-ordinate systems we have to deal with.

Firstly, we have the world co-ordinate systems (xw,yw,zw). Since the camera may move, or we may use multiple camearas, it can hardly serve as a fixed frame of reference, and the world frame should be independent of the camera.

Secondly, we have the camera frame, capital (X,Y,Z), with the Z-axis along the optical axis. The Y-axis points down to make a right hand system.

Thirdly, we have the image frame. The lower-case (x,y) co-ordinates denote a point in the image plane with origin on the optical axis.

Finally, there is a fourth co-ordinate system. When we sample the image and represent the pixmap as an array, we index the elements (i,j) from the upper left-hand corner. The first co-ordinate i counts the rows, and the second co-ordinate counts columns.

The three first co-ordinate systems are index using real numbers, while the fourth one is indexed in integers, since the pixmap does not allow for fractional pixels.

The first goal of camera calibration is to determine all the co-ordinate transformations between these four frames.

Point $P$ has co-ordinates

$\vec{x}_w=(x_w,y_w,z_w)$ in the world frame
$\vec{X}=(X,Y,Z)$ in the camera frame
$\vec{x}=(x,y)$ in the image plane
$(i,j)$ in the pixmap

Seek mappings $$\vec{x}_w\mapsto \vec{X}\mapsto \vec{x}\mapsto (i,j)$$

World frame and camera frame

Change of basis	$$\mathbf{X} = R\mathbf{x}_w + T_{O_w} \in\mathbb{R}^3$$
where	$R = \begin{bmatrix} \mathbf{e}_{w,1} \mathbf{e}_{w,2} \mathbf{e}_{w,3} \end{bmatrix}$
or with homogeneous co-ordinates	$$\tilde{\mathbf{X}} = \begin{bmatrix} R & T_{O_w} \\ 0 & 1 \end{bmatrix}\cdot \tilde{\mathbf{x}}_w$$

Projection

ideal perspective	$$\mathbf{x} = \begin{bmatrix}x\\y\end{bmatrix} = \frac fZ\begin{bmatrix}X\\Y\end{bmatrix}$$
homogeneous	$$ Z \begin{bmatrix}x\\y\\1\end{bmatrix} = \begin{bmatrix}f & 0 & 0 & 0 \\0 & f & 0 & 0 & \\0 & 0 & 1 & 0 \end{bmatrix} \cdot \begin{bmatrix}X\\Y\\Z\\1\end{bmatrix}$$
$Z$ is unknown. Write $\lambda (=Z)$	$$ \lambda \begin{bmatrix}x\\y\\1\end{bmatrix} = \begin{bmatrix}f & 0 & 0 & 0 \\0 & f & 0 & 0 \\0 & 0 & 1 & 0 \end{bmatrix} \cdot \begin{bmatrix}X\\Y\\Z\\1\end{bmatrix}$$

The second step is from the camera frame to the image plane, and we recall the ideal perspective from the camera model. We find the lower case (x,y) by scaling down the upper case (X,Y) by a factor of f divided by Z. In other words, we move the point along the ray through the origin until it falls inside the image plane.

If we write it all in homegeneous co-ordinates, also the Z co-ordinate has to be divided by Z. Thus it becomes one, and the image point is written as (x,y,1) which is valid homogeneous co-ordinates in 2D.

The tricky part is that capital Z is unknown. The observed point may be anywhere on the line through (x,y) and the origin. To emphasise its role as an unknown constant, we write lambda for Z. Later in the module, we will see how stereo vision allows us to calculate lambda by triangulation.

Ideal Camera Projection

$$ \lambda \begin{bmatrix}x\\y\\1\end{bmatrix} $$	$$ = $$	$$\begin{bmatrix}f & 0 & 0 \\0 & f & 0 \\0 & 0 & 1 \end{bmatrix}$$	$$\cdot$$	$$\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0\end{bmatrix} $$	$$\cdot$$	$$ \begin{bmatrix}R & T \\0 & 1\end{bmatrix}$$	$$\cdot$$	$$\begin{bmatrix}X\\Y\\Z\\1\end{bmatrix}$$
$$ \lambda \textbf{x} $$	$$ = $$	$$K_f$$	$$\cdot$$	$$\Pi_0$$	$$\cdot$$	$$g$$	$$\cdot$$	$$\mathbf{X}_0$$

From metres to pixels

pixels per metric unit	$s_x\times s_y$
axes scaled differently	$$\begin{bmatrix} x_s\\y_s\end{bmatrix} = \begin{bmatrix} s_x & 0\\ 0 & s_y\end{bmatrix} \cdot \begin{bmatrix} x\\y\end{bmatrix} $$
translate the origin	$$\begin{align}i &= x_s + o_x \\ j &= y_s + o_y\end{align} $$
homogeneous co-ordinates	$$\begin{bmatrix} i\\j\end{bmatrix} = \begin{bmatrix} s_x & 0 & o_x \\ 0 & s_y & o_y \\ 0 & 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} x\\y\\1\end{bmatrix} $$

non-rectangular grid

$$\begin{bmatrix} i\\j\\1\end{bmatrix} = \begin{bmatrix} s_x & s_\theta & o_x \\ 0 & s_y & o_y \\ 0 & 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} x\\y\\1\end{bmatrix} $$

Camera Intrinsic Parameters

$$ \lambda \begin{bmatrix}x\\y\\1\end{bmatrix} $$	$$ = $$	$$\begin{bmatrix} s_x & s_\theta & o_x \\ 0 & s_y & o_y \\ 0 & 0 & 1 \end{bmatrix}$$	$$\cdot$$	$$\begin{bmatrix}f & 0 & 0 \\0 & f & 0 \\0 & 0 & 1 \end{bmatrix}$$	$$\cdot$$	$$\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0\end{bmatrix} $$	$$\cdot$$	$$\begin{bmatrix}X\\Y\\Z\\1\end{bmatrix}$$
$$ \lambda \textbf{x} $$	$$ = $$	$$K_s$$	$$\cdot$$	$$K_f$$	$$\cdot$$	$$\Pi_0$$	$$\cdot$$	$$\mathbf{X}_0$$

Camera Intrinsic Parameter Matrix	$$K=K_sK_f= \begin{bmatrix} fs_x & fs_\theta & o_x \\ 0 & fs_y & o_y \\ 0 & 0 & 1 \end{bmatrix}$$