---
title: Multiscale Detection
categories: lecture
---

**Reading** Ma 2004 Chapter 11.2

# Principle

1. Consider motion tracking by differentiation.
2. Differentiation is based on infinitesimaly small changes.
   In practice it may work for motion up to 2-3 pixels/frame.
3. Advantage: can track subpixel movement.
4. Problems for
    - fast movement
    - high resolution
    - widely different camera angles
5. For widely different camera angles we need to look elsewhere,
   but for fast movement and high resolution, we can use
   *multiscale tracking*
6. **Example** Suppose the movement is 8 pixels/frame, clearly too much.
    - Downsample the image by a factor of ¼.
    - I.e. reduce the resolution by a quarter.
    - What is the movement rate now?

# Multiscale Pyramid

1. Two input frames $I_1$ and $I_2$.
2. Write $I_i^1=I_i$.
3. Form $I^k_i$ by downsampling $I^{k-1}_i$ by a factor of ½.
4. Two to four scales is normally sufficient
5. **Example** Starting with frames at $800\times640$,
   we form pyramids with versions with resolutions
   $800\times640$, $200\times160$, $400\times320$, and $100\times80$.
6. Smoothing is used at each downsampling step

# Procedure

## Coarsest level

1. Start with the coarsest level, that is the largest value of $k$.
2. Compare $I_1^k$ and $I_2^k$ to calculate the motion $d^k$ as usual.
3. Maintain the running total estimated motion $d\leftarrow d^k$.

## Subsequent levels

1. Consider level $k$.  We have estimated the motion in level $k+1$
   as $d$.  This corresponds to a motion of $2d$ in level $k$.
2. Comparing $I_1^k$ and $I_2^k$  may not work, because the movement
   is too large.  Therefore we adjust one iage to compensate for
   known motion.
3. Compute $\tilde I_2^k$ by shifting $I_2^k$ $2d$ pixels, i.e.
   $\tilde I_2^k=I_2^k(x+2d)$.
4. Using $I_1^k$ and $\tilde I_2^k$ we estimate the motion $d^k$
   as before.
5. The total estimated motion is updated $d\leftarrow 2d+d^k$.
   That is, we add the motion estimated to the motion removed.

Beware the sign of the movement.  Ma (Eq 11.2) computes
movement from image 2 to image 1, therefore we have a plus
$x+2d$ when we calculate $\tilde I$.  It is important to 
check that we have the movement in the right direction.

## End of stage 1

1. When we have completed level $k=1$ we have the total estimated
   motion as $d=d^1+2d^2+\cdots+2^{k-1}d^k$.

## Stage 2. Fine tuning

1. Fine tuning is done at the finest scale ($I^1_1,I^1_2$).
2. Make the image $\tilde I_2=I_2(x+d)$.
3. If $d$ is correctly estimated, these two images should be equal.
4. They probably are not.
5. Estimate the estimation error $\Delta d$ as movement for the
   pair $I_1,\hat I_2$.
6. Adjust the estimate $d\gets d+\Delta d$.
7. Repeat from Step 5 until the correction term is small enough
   (or you it diverges and you conclude that this does not work).