# Image Formation

Vision is the inverse problem of image formation

This session is prepared for **self-study**. However, the room is available, and you should meet up and take advantage of collaboration.

Use the **first hour** to watch and listen to the lectures (videos and slideshows with audio). See under Briefing. Use the **rest of the time** to solve the Exercises as group work. You should discuss possible solutions between yourselves **before** you review the Solutions. Remember that communicating and arguing solutions to your peers is one of the most important learning outcomes of the module.

# Briefing

The actual briefing will extensively use blackboard drawings and improvisation. Hence the lecture notes below are **not complete**.

- Image Formation Lecture
- Rudimentary notes from 2022: Image Formation Notes and Slides

## Learning Outcomes

During this session, the goal is to learn to master the following concepts and models:

- The image as a sampled function
- Projection from 3D to 2D, as it occurs in a camera
- thin lens equation
- vanishing point
- The thin lens model
- aperture, focus
- The pinhole model

# Exercises

Exercises are from Ma 2004 page 62ff.

I recommend to discuss the following problems in small groups. Use figures and diagrams as basis for your discussion where possible.

If you prefer, you may consult the Solutions after each individual exercise.

## Equivalence of Points (Based on Exercise 3.1.)

Show that any point on the line through \(o\) (optical centre) and \(p\) projects onto the same image co-ordinates as \(p\).

- Start by drawing the lens, image, the points \(p\) and \(o\), and the image point.
- What does the drawing tell you about the problem? Add details to the drawing as required.
- Recall the equations which relate the \((x,y)\) co-ordinates of the image point to the \((X,Y,Z)\) co-ordinates of \(p\). (Write it down.)
- Consider a different point \(p'\) on the same line, and add it to your drawing. Where is its image point?
- How does do the co-ordinates \((X',Y',Z')\) of \(p'\) relate to \((X,Y,Z)\) and \((x,y)\)?
- From the above, you should have two arguments solving the problem, one geometric and one algebraic. Each deserves attention. Are these arguments convincing? Complete any details as required.
- Reflect on the relationship between the algebraic and the geometric argument.

## (Exercise 3.2)

Consider a thin lens imaging a plane parallel to the lens at a distance \(z\) from the focal plane. Determine the region of this plane that contributes to the image \(I\) at the point \(x\). (Hint: consider first a one-dimensional imaging model, then extend to a two-dimensional image.)

**Note** The question makes sense if you assume that the plane is out of focus, which is not possible in the pinhole model but is in a more generic thin lens model.

- Always start by making a drawing of the model.
- Add all concepts mentioned in the problem text to the figure (as far as possible).
- Add any additional concepts that you find important.
- Identify the concept in question, that is the region contributing to the point \(x\) in this case.

## Scale Ambiguity (Exercise 3.8).

It is common sense that with a perspective camera, one cannot tell an object from another object that is exactly twice as big but twice as far. This is a classic ambiguity introduced by the perspective projection. Use the ideal camera model to explain why this is true. Is the same also true for the orthographic projection? Explain.

- You can start with the problem you drew above for Exercise 1 (Ma:3.1). Consider an object extending between two points \(p_1\) and \(p_2\) in a plane parallel to the lens. Draw this situation.
- Imagine that both points move on a line through the optical centre \(o\), as you did in Exercise 1. What happens to the image? What happens to the object extending between \(p_1\) and \(p_2\)?
- Write up an argument based on the above reflections.

## Field of View (based on Exercise 3.3 Part 1)

How can describe the area (in 3D) observed by a camera?

Consider a camera with focal length 24 mm, and a retinal plane (CCD array) (16 mm x 12 mm).

- As always, start with a drawing. Draw the pinhole module. Consider only the \(x\)-direction where the sensor is 16 mm. (You can do the \(y\)-direction (12mm) afterwards.)
- Write the known lengths into the figure.
- Where are the points which are observable to the camera? Reflect on the question.
- You should find that the observable points fall betweeen two lines through the focus (pinhole). Calculate the angle \(\theta\) between these two lines.
- Note that the optical axis, through the focus, is orthogonal and centred on the sensor array. It may be easier to calculate the angle \(\theta/2\) between the optical axis and one of the edge lines.

- The angle \(\theta\) is known as the field of view (FoV). Once you have calculated FoV for the specific camera, give an expression of FoV as a function of the focal length \(f\) and the radius of the sensor \(r\).

## Real World and Imavge Co-ordinates (based on Exercise 3.3 Part 2)

Given a point \((X,Y,Z)\) in 3D, what is the co-ordinates \((x,y)\) of the image point?

Consider the same camera system and model as you used in the previous exercise. Consider first a point with co-ordinates \((X,Y,Z)=(6m,4m,8m)\).

- Draw first the pinhole model in the \(x\)-direction and find the \(x\)-co-ordinate corresponding to \(X=6m\).
- Then draw the \(xy\)-direction and find the \(y\)-co-ordinate corresponding to \(X=4m\).

# Debrief

See Solutions