Image Formation

Machine Vision

Hans Georg Schaathun

NTNU, Noregs Teknisk-Naturvitskaplege Universitet

Fri 20 Aug 06:55:55 UTC 2021

  1. Briefing Lens models
  2. Exercise
  3. Debrief

Vision

Eye Model from Introduction to Psychology by University of Minnesota

  • 2D projection on the Retina.
  • Modern Cameras replicate the Eye Model.
  • Fine grid of sensors perceives rays of light
Vision is the inverse problem of image formation

The Mandrill Test Image (public domain) 

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt

im = cv.imread("mandrill-grey.png")

The Mandrill Test Image (public domain) 

In [5]: im
Out[5]:
array([[[ 76,  76,  76],
        [ 60,  60,  60],
        [ 68,  68,  68],
        ...,
        [ 90,  90,  90],
        [ 97,  97,  97],
        [ 96,  96,  96]],
        
       [[ 83,  83,  83],
        [ 72,  72,  72],
        [ 77,  77,  77],
        ...,
        [ 99,  99,  99],
        [ 87,  87,  87],
        [106, 106, 106]],
        
        ...,

       [[131, 131, 131],
        [144, 144, 144],
        [138, 138, 138],
        ...,
        [ 85,  85,  85],
        [ 98,  98,  98],
        [ 90,  90,  90]],
        
       [[109, 109, 109],
        [102, 102, 102],
        [109, 109, 109],
        ...,
        [ 57,  57,  57],
        [ 67,  67,  67],
        [ 69,  69,  69]]], dtype=uint8)
        
In [6]: im.shape
Out[6]: (128, 128, 3)

In [7]:
  • confusing: three dimensions, as if RGB
  • grey scale only needs $128\times128$ matrix
  • equal values in each tripple
In [7]: im[:,:,0]
Out[7]: 
array([[ 76,  60,  68, ...,  90,  97,  96],
       [ 83,  72,  77, ...,  99,  87, 106],
       [ 51,  75, 117, ...,  99,  81,  88],
       ...,
       [139, 140, 136, ...,  92,  96,  78],
       [131, 144, 138, ...,  85,  98,  90],
       [109, 102, 109, ...,  57,  67,  69]], dtype=uint8)
       
In [8]:

The Mandrill Test Image (public domain) 

In [60]: cv.imshow("mandrill grey", im)

In [61]: cv.waitKey(1)
Out[61]: -1

Sampled function $I(x,y)$

Out[58]:
array([[ 76,  60,  68, ...,  90,  97,  96],
       [ 83,  72,  77, ...,  99,  87, 106],
       [ 51,  75, 117, ...,  99,  81,  88],
       ...,
       [139, 140, 136, ...,  92,  96,  78],
       [131, 144, 138, ...,  85,  98,  90],
       [109, 102, 109, ...,  57,  67,  69]], dtype=uint8)
       
In [59]:
  • Upside down Is $(0,0)$ the top or lower left hand corner?
  • Always Sampled
3D surface plot of the image signal

Thin Lens Model

Burning Glass from *College Physics* by OpenStax College

  • parallel rays meet in the focus

Thin Lens Model

Diagram of Lens Focus Point from *College Physics* by OpenStax College

  • Optical Axis is the line perpendicular on the lens, through its centre.
  • The Focus is a point on the Optical Axis. Rays which enter the lense parallel to the optical axis are deflected so that they intersect at the Focus.
  • Focal Length is the distance between the lens and the Focus.
  • The Focal Plane is a plane through the Focus, perpendicular on the Optical Axis.

The Image Plane

Image of points (first lens model)

  • a sensor behind the focus point catches the projection
  • point $p$ (resp. $q$) gives an image $p'$ (resp. $q'$).

The aperture

Image of points (first lens model)

  • one ray gives too little light
  • larger aperture gives more light
  • many rays into one point $\Rightarrow$ better sensing

Thin Lens Equation

  • Origo Optical centre, i.e. the intersection of the lens and the optical axis.
  • The optical axis is the $z$-axis.
  • Say $P$ is an object point with co-ordinates $(X,Y,Z)$
  • and $p$ is the corresponding image point with co-ordinates $(x,y,z)$

$$\frac1Z + \frac1z = \frac1f$$

Draw figure

$$\frac Z X = \frac z x\quad\text{and}\quad \frac{f}{X} = \frac{z-f}{x}$$

$$\frac x X = \frac z Z\quad\text{and}\quad \frac{x}{X} = \frac{z-f}{f}$$

$$\frac{z}{Z} = \frac z f-1$$

$$\frac{1}{Z} = \frac1f-\frac{1}{z}$$

$$\frac{1}{Z} + \frac{1}{z} = \frac1f$$

Out of Focus

Image of points - out of focus

  • large aperture has a disadvantage.
  • $p$ and $q$ had just the right distance from the lens
  • $r$ is closes - rays do not intersect

The pinhole model

  • lens radius goes to zero
  • the pinhole only admits rays through the focus point itself.
  • Consequence: any point in focus
  • no parallel rays - no focus point
  • New figure

Ideal Perspective Projection

Refer to Pinhole Figure

  • $\mathbb{R}^3 \to \mathbb{R}^2$
  • $(X,Y,Z) \mapsto (x,y) = \bigg(-f\frac{X}{Z},-f\frac{Y}{Z}\bigg)$

Frontal Pinhole Model

  • Avoid the signs
  • $(X,Y,Z) \mapsto (x,y) = \bigg(f\frac{X}{Z},f\frac{Y}{Z}\bigg)$

Perspective

Perspective Bias

Which man is larger in the drawing?

Vanishing points

Field of Vision

  • $$\theta = 2\tan^{-1}\frac{r}{f}$$
  • where $2r$ is the diameter of the sensor
  • Draw Figures