Basic feature tracking: Algorithm 4.1 (Ma 2004)

(This is Ma’s algorithm for feature tracking only. It has been copy-edited to remove cross-references and references to optical flow.)

Given an image \(I(x)\) at time \(t\), set a window \(W\) of fixed size.

1. Calculate the image gradients \((I_x,I_y)\) using a differentiation filter, for instance the Sobel filter.
2. Calculate the time difference \(I_t\) as the difference between the current frame \(I(x)\) and the subsequent or previous frame.
3. Compute \[ G(x) = \begin{bmatrix} \sum I_x^2 & \sum I_xI_y \\ \sum I_xI_y & \sum I_y^2 \end{bmatrix} \] at every pixel \(\mathbf{x}\) The sums are taken over the window \(W\).
4. Select a number of point features by choosing \(\mathbf{x}_1,\mathbf{x}_2,\ldots\) such that \(G(X_i)\) is invertible.
5. Compute \[\mathbf{b}(\mathbf{x}, t) = \begin{bmatrix} \sum I_xI_t \\ \sum I_yI_t \end{bmatrix}\] The sums are taken over the window \(W\).
6. Compute the displacement \(\mathbf{u}(x, t)\) as \[ \mathbf{u}(\mathbf{x}, t) = -G(\mathbf{x})^{-1}\mathbf{b}(\mathbf{x}, t)\]
7. At time \(t + 1\), repeat the operation at \(\mathbf{x} + \mathbf{u}(\mathbf{x} , t)\).