Revision ab5a1b82f0473bef22605732b05b9b6273ca3a15 (click the page title to view the current version)
Changes from ab5a1b82f0473bef22605732b05b9b6273ca3a15 to 4276c443c1b243d8e02d1d536ad08ae77412f55e
---
title: Tracking Features
categories: session
---
**Date** 23 September 2021
- aperture problem [page 78]
**Briefing** [Tracking Features Lecture]()
# Briefing
# Exercises
## Corners
+ Exercises 4.1, 4.2, 4.3 (Ma 2004)
+ (Exercises 4.4 (Ma 2004))
# Debrief
+ What are distinctive points in the image?
+ Distinctive points can (to some extent) be matched in two different images.
[More Images](https://www.flickr.com/photos/ntnu-trondheim/collections/72157632165205007/)
## Corner Correspondence
+ Two images of the same scene
$I_1,I_2: \Omega\subset\mathbb{R}^3\to\mathbb{R}_+ ; \mathbf{x}\mapsto I_1(\mathbf{x}),I_2(\mathbf{x})$
+ Different in general
*Why are they different?*
## Brightness Constancy Constraint
+ Suppose we photograph empty space except for a single point $p$
- *Brightness Constancy Constraint*
$$I_1(\mathbf{x}_1) = I_2(\mathbf{x}_2) \sim \mathcal{R}(p)$$
+ Simple dislocation from $\mathbf{x}_1$ to $\mathbf{x}_2$
+ Problem: Globally, it is an infinite-dimentional transformation
+ Motion: $h: \mathbf{x}_1\mapsto\mathbf{x}_2$
+ so that $I_1(\mathbf{x}_1) = I_2(h(\mathbf{x}_1)) \forall
\mathbf{x}_1\in\Omega\cap h^{-1}(\Omega)\subset\mathbb{R}^{2\times2}$
## Motion Models
+ Affine Motion Model: $h(\mathbf{x}_1) = A\mathbf{x}_1 + \mathbf{d}$
+ Projective Motion Model: $h(\mathbf{x}_1) = H\mathbf{x}_1$ where
$H\in\mathbb{R}^{3\times3}$ is defined up to a scalar factor.
+ Need to accept changes to the intencity
## Aperture Problem
$$\hat h = \arg\min_h\sum_{\tilde\mathbf{x}\in W(\mathbf{x})} ||I_1(\tilde\mathbf x)-I_2(\tilde\mathbf x)||^2$$
+ The window, or aperture, $W(\vec{x})$
+ cannot distinguish points on a blank wall
+ Choose $h$ from a family of functions, parameterised by $\alpha$
+ translational: $\alpha=\Delta\mathbf{x}$
+ affine: $\alpha=\{A,\mathbf{d}\}$
## Feature Tracking
$$I_1(\textbf{x})= I_2(h(\textbf{x}))= I_2(\textbf{x}+\Delta\textbf{x})$$
+ Consider infitesimally small $\Delta\textbf x$
## Infinitesimal Model
+ Model on a time axis - two images taken infinitesimally close in time
+ ... under motion
$$I(\mathbf{x}(t),t) = I(\mathbf{x}(t)+t\mathbf{u},t+dt)$$
$$\nabla I(\mathbf{x}(t),t)^\mathrm{T}\mathrm{u} + I_t(\mathbf{x}(t),t) = 0$$
$$\nabla I(\mathbf{x},t) = \begin{bmatrix} I_x(\mathbf{x},t)\\ I_y(\mathbf{x},t) \end{bmatrix}
= \begin{bmatrix}\frac{\partial I}{\partial x}(\mathbf{x},t)\\ \frac{\partial I}{\partial y}(\mathbf{x},t) \end{bmatrix}
\in\mathbb{R}^2$$
$$I_t(\mathbf{x},t) = \frac{\partial I}{\partial t}(\mathbf{x},t)\in \mathbb{R}$$
*Brightness Constancy Constraint* for the simplest possible continuous model
+ Two applications
- optical flow: fix a position $\mathbf x$ and consider particles passing through
- feature tracking: fix a partical $x(t)$ an track it through space
## Solving for $\textbf{u}$
$$\nabla I^\mathrm{T}\mathrm{u} + I_t = 0$$
+ There are infititly many solutions, due to the *aperture problem*
+ We can solve for the component in the direction of the gradient though
$$\frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||} = - \frac{I_t}{||\nabla I||} $$
+ Left hand side is the scalar projection of $\mathbf u$ onto $\nabla I$.
+ Multiplying by $\nabla I/||\nabla I||$, we get the vector projection:
$$\mathbf u_n = \frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||}\cdot\frac{\nabla I}{||\nabla I||} = - \frac{I_t}{||\nabla I||\cdot\frac{\nabla I}{||\nabla I||}} $$
