Revision 1443d4fe1561bd2be759f5ad44c79b600304c488 (click the page title to view the current version)
Tracking Features
- aperture problem [page 78]
Briefing
Corners
- What are distinctive points in the image?
- Distinctive points can (to some extent) be matched in two different images.
Corner Correspondence
- Two images of the same scene \(I_1,I_2: \Omega\subset\mathbb{R}^2\to\mathbb{R}_+ ; \mathbf{x}\mapsto I_1(\mathbf{x}),I_2(\mathbf{x})\)
- Different in general
Why are they different?
- Firstly - different colour in different directions
- Lambertian assumption
- Secondly - noise
- let’s assume this is insignificant
- Thirdly - different image points \(\mathbf{x}_1,\mathbf{x}_2\) correspond to the same 3D point \(p\)
Brightness Constancy Constraint
- Suppose we photograph empty space except for a single point \(p\)
- Brightness Constancy Constraint
\[I_1(\mathbf{x}_1) = I_2(\mathbf{x}_2) \sim \mathcal{R}(p)\]
- Simple dislocation from \(\mathbf{x}_1\) to \(\mathbf{x}_2\)
- Motion: \(h: \mathbf{x}_1\mapsto\mathbf{x}_2\) so that \[\forall \mathbf{x}_1\in\Omega\cap h^{-1}(\Omega)\subset\mathbb{R}^{2}, \;I_1(\mathbf{x}_1) = I_2(h(\mathbf{x}_1))\]
Motion Models
- Translational Motion Model: \[h(\mathbf{x}_1) = \mathbf{x}_1 + \mathbf{\Delta x}\]
- Affine Motion Model: \[h(\mathbf{x}_1) = A\mathbf{x}_1 + \mathbf{d}\]
- Projective Motion Model: \[h(\mathbf{x}_1) = H\mathbf{x}_1\] where \(H\in\mathbb{R}^{3\times3}\) is defined up to a scalar factor.
Intencity Transformation
- Need to accept changes to the intencity
\[I_1(\mathbf{x}_1) = I_2(h(\mathbf{x}_1)) + n(h(\mathbf{x}_1))\]
- Occlusions
- Non-Lambertian reflection
- Taken at different time? Different ambient light?
Aperture Problem
Estimator \[\hat h = \arg\min_h\sum_{\tilde{\mathbf{x}}\in W(\mathbf{x})} ||I_1(\tilde{\mathbf{x}})-I_2(h(\tilde{\mathbf{x}}))||^2\]
- The window, or aperture, \(W(\vec{x})\)
- Choose \(h\) from a family of functions, parameterised by \(\alpha\)
- translational: \(\alpha=\Delta\mathbf{x}\)
- affine: \(\alpha=\{A,\mathbf{d}\}\)
Aperture problem: cannot distinguish points on a blank wall
Feature Tracking
\[I_1(\textbf{x})= I_2(h(\textbf{x}))= I_2(\textbf{x}+\Delta\textbf{x})\]
- Consider infitesimally small \(\Delta\textbf x\)
Infinitesimal Model
- Model on a time axis - two images taken infinitesimally close in time
- … under motion
\[I(\mathbf{x}(t),t) = I(\mathbf{x}(t)+t\mathbf{u},t+dt)\]
\[\nabla I(\mathbf{x}(t),t)^\mathrm{T}\mathrm{u} + I_t(\mathbf{x}(t),t) = 0\]
\[\nabla I(\mathbf{x},t) = \begin{bmatrix} I_x(\mathbf{x},t)\\ I_y(\mathbf{x},t) \end{bmatrix} = \begin{bmatrix}\frac{\partial I}{\partial x}(\mathbf{x},t)\\ \frac{\partial I}{\partial y}(\mathbf{x},t) \end{bmatrix} \in\mathbb{R}^2\]
\[I_t(\mathbf{x},t) = \frac{\partial I}{\partial t}(\mathbf{x},t)\in \mathbb{R}\]
Brightness Constancy Constraint for the simplest possible continuous model
- Two applications
- optical flow: fix a position \(\mathbf x\) and consider particles passing through
- feature tracking: fix a partical \(x(t)\) an track it through space
Solving for \(\textbf{u}\)
\[\nabla I^\mathrm{T}\mathrm{u} + I_t = 0\]
- There are infititly many solutions, due to the aperture problem
- We can solve for the component in the direction of the gradient though
\[\frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||} = - \frac{I_t}{||\nabla I||} \]
- Left hand side is the scalar projection of \(\mathbf u\) onto \(\nabla I\).
- Multiplying by \(\nabla I/||\nabla I||\), we get the vector projection:
\[\mathbf u_n = \frac{\nabla I^\mathrm{T}\mathrm{u}}{||\nabla I||}\cdot\frac{\nabla I}{||\nabla I||} = - \frac{I_t}{||\nabla I||\cdot\frac{\nabla I}{||\nabla I||}} \]