Image Formation

Vision is the inverse problem of image formation

Briefing

The actual briefing will extensively use blackboard drawings and improvisation. Hence the lecture notes below are not complete.

Learning Outcomes

During this session, the goal is to learn to master the following concepts and models:

  • The image as a sampled function
  • Projection from 3D to 2D, as it occurs in a camera
  • thin lens equation
  • vanishing point
  • The thin lens model
  • aperture, focus
  • The pinhole model

Exercises

Exercises are from Ma 2004 page 62ff.

I recommend to discuss the following problems in small groups. Use figures and diagrams as basis for your discussion where possible.

Equivalence of Points (Based on Exercise 3.1.)

Show that any point on the line through \(o\) (optical centre) and \(p\) projects onto the same image co-ordinates as \(p\).

  1. Start by drawing the lens, image, the points \(p\) and \(o\), and the image point.
  2. What does the drawing tell you about the problem? Add details to the drawing as required.
  3. Recall the equations which relate the \((x,y)\) co-ordinates of the image point to the \((X,Y,Z)\) co-ordinates of \(p\). (Write it down.)
  4. Consider a different point \(p'\) on the same line, and add it to your drawing. Where is its image point?
  5. How does do the co-ordinates \((X',Y',Z')\) of \(p'\) relate to \((X,Y,Z)\) and \((x,y)\)?
  6. From the above, you should have two arguments solving the problem, one geometric and one algebraic. Each deserves attention. Are these arguments convincing? Complete any details as required.
  7. Reflect on the relationship between the algebraic and the geometric argument.

(Exercise 3.2)

Consider a thin lens imaging a plane parallel to the lens at a distance \(z\) from the focal plane. Determine the region of this plane that contributes to the image \(I\) at the point \(x\). (Hint: consider first a one-dimensional imaging model, then extend to a two-dimensional image.)

Note The question makes sense if you assume that the plane is out of focus, which is not possible in the pinhole model but is in a more generic thin lens model.

  1. Always start by making a drawing of the model.
  2. Add all concepts mentioned in the problem text to the figure (as far as possible).
  3. Add any additional concepts that you find important.
  4. Identify the concept in question, that is the region contributing to the point \(x\) in this case.

Scale Ambiguity (Exercise 3.8).

It is common sense that with a perspective camera, one cannot tell an object from another object that is exactly twice as big but twice as far. This is a classic ambiguity introduced by the perspective projection. Use the ideal camera model to explain why this is true. Is the same also true for the orthographic projection? Explain.

  1. You can start with the problem you drew above for Exercise 1 (Ma:3.1). Consider an object extending between two points \(p_1\) and \(p_2\) in a plane parallel to the lens. Draw this situation.
  2. Imagine that both points move on a line through the optical centre \(o\), as you did in Exercise 1. What happens to the image? What happens to the object extending between \(p_1\) and \(p_2\)?
  3. Write up an argument based on the above reflections.

Exercise 3.3 Part 1-2.

Part 3-4 depends on the camera calibration which we discuss in the next session.

An important parameter of the imaging system is the field of view (FOV). The field of view is twice the angle between the optical axis (z-axis) and the end of the retinal plane (CCD array). Imagine having a camera system with focal length 24 mm, and retinal plane (CCD array) (16 mm x 12 mm) and that your digitizer samples your imaging surface at 500 x 500 pixels in the horizontal and vertical directions.

  1. Compute the FOV.
  2. Write down the relationship between the image coordinate and a point in 3-D space expressed in the camera coordinate system.
  1. As before, make first the drawing. It is similar to the ones you have made already.
  2. Put all the measurements into the figure.
  3. How can you calculate the angle?
    Use the horizontal dimension of the sensor, and find a straight-angled triangle where you know two sides and where the angle of interest occurs.
  4. To calculate the image co-ordinates, it is helpful to draw 2D cuts, and consider the \(x\)- and \(y\)-co-ordinates separately.

Debrief

  1. Solutions
  2. Questions and Answers
  3. Recap as required