• aperture problem [page 78]

Briefing

Corners

• What are distinctive points in the image?
• Distinctive points can (to some extent) be matched in two different images.

More Images

Corner Correspondence

• Two images of the same scene \(I_1,I_2: \Omega\subset\mathbb{R}^3\to\R_+ ; \mathbf{x}\mapsto I_1(\mathbf{x}),I_2(\mathbf{x})\)
• Different in general

Why are they different?

Brightness Constancy Constraint

• Suppose we photograph empty space except for a single point \(p\)
  • Brightness Constancy Constraint

\[I_1(\mathbf{x}_1) = I_2(\mathbf{x}_2) \sim \mathcal{R}(p)\]

• Simple dislocation from \(\mathbf{x}_1\) to \(\mathbf{x}_2\)
• Problem: Globally, it is an infinite-dimentional transformation
• Motion: \(h: \mathbf{x}_1\mapsto\mathbf{x}_2\)
  • so that \(I_1(\mathbf{x}_1) = I_2(h(\mathbf{x}_1)) \forall \mathbf{x}_1\in\Omega\cap h^{-1}(\Omega)\subset\mathbb{R}^{2\times2}\)

Motion Models

• Affine Motion Model: \(h(\mathbf{x}_1) = A\mathbf{x}_1 + \mathbf{d}\)
• Projective Motion Model: \(h(\mathbf{x}_1) = H\mathbf{x}_1\) where \(H\in\mathbb{R}^{3\times3}\) is defined up to a scalar factor.
• Need to accept changes to the intencity

Aperture Problem

\[\hat h = \arg\min_h\sum_{\tilde\mathbf{x}\in W(\mathbf{x})} ||I_1(\tilde\mathbf x)-I_2(\tilde\mathbf x)||^2\]

• The window, or aperture, \(W(\vec{x})\)

• cannot distinguish points on a blank wall

• Choose \(h\) from a family of functions, parameterised by \(\alpha\)
  • translational: \(\alpha=\Delta\mathbf{x}\)
  • affine: \(\alpha=\{A,\mathbf{d}\}\)

Feature Tracking

\[I_1(\textbf{x})= I_2(h(\textbf{x}))= I_2(\textbf{x}+\Delta\textbf{x})\]

• Consider infitesimally small \(\Delta\textbf x\)

Infinitesimal Model

• Model on a time axis - two images taken infinitesimally close in time
  • … under motion

\[I(\mathbf{x}(t),t) = I(\mathbf{x}(t)+t\mathbf{u},t+dt)\]

\[\nabla I(\mathbf{x}(t),t)^\mathrm{T}\mathrm{u} + I_t(\mathbf{x}(t),t) = 0\]

\[\nabla I(\mathbf{x},t) = \begin{bmatrix} I_x(\mathbf{x},t)\\ I_y(\mathbf{x},t) \end{bmatrix} = \begin{bmatrix}\frac{\partial I}{\partial x}(\mathbf{x},t)\\ \frac{\partial I}{\partial y}(\mathbf{x},t) \end{bmatrix} \in\mathbb{R}^2\]

\[I_t(\mathbf{x},t) = \frac{\partial I}{\partial t}(\mathbf{x},t)\in \mathbb{R}\]

Brightness Constancy Constraint for the simplest possible continuous model

• Two applications
  • optical flow: fix a position \(\mathbf x\) and consider particles passing through
  • feature tracking: fix a partical \(x(t)\) an track it through space

Solving for \(\textbf{u}\)

\[\nabla I^\mathrm{T}\mathrm{u} + I_t = 0\]

• There are infititly many solutions, due to the aperture problem
• We can solve for the component in the direction of the gradient though

\[\frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||} = - \frac{I_t}{||\nabla I||} \]

• Left hand side is the scalar projection of \(\mathbf u\) onto \(\nabla I\).
• Multiplying by \(\nabla I/||\nabla I||\), we get the vector projection:

\[\mathbf u_n = \frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||}\cdot\frac{\nabla I}{||\nabla I||} = - \frac{I_t}{||\nabla I||\cdot\frac{\nabla I}{||\nabla I||}} \]