Camera Calibration

Briefing

Under construction Links may not yet work.

The goal today is to understand what camera calibration is, why we need it, and how to do it in OpenCV (python). The exercise is a standard tutorial, which should be straight forward. To understand what calibration is, we need first to note that there are two kinds. It is easy to get confused, because many authors consider only one kind, depending on context, without making it explicit.

I have prepared two talks to explain this:

  • Linear Calibration assumes ideal projection and aims to determine all the details of the mapping from 3D to 2D.
  • Non-Linear Distortion is the result of imperfect lenses which make the camera model inaccurate. The projection is no longer ideal.

Lecture Notes on Camera Mathematics from previous years are still available, but I think the talks above are easier to understand.

Exercises

We are going to do the calibration in practice. Note that …

  • OpenCV does not detect corners in all images and the tutorial recommends running a loop searching for corners while you move the chessboard
  • You may have to tweak the delays so that you see what you are doing in real time. I had to fiddle a lot to get this right.

Exercise 1. Tutorial

  1. Complete the tutorial: Calibration in OpenCV
  2. Discuss the following questions, using the hints below and the OpenCV documentation.
    • Where in this tutorial do you calculate the camera intrinsic matrix?
    • How can you find the radial distortion parameters?
    • What can you say about the other variables calculated in the tutorial?

The most critical calibration functions are the following.

The output parameters are explained in the reference documentation. However, the documentation is written for the C++ syntax. Hence, most of the output parameters will appear as input parameters in the documentation. In the code example above, I have used the same parameter names as in the C++ documentation.

When the image is transformed to remove distortion, the result is not rectangular. One has to choose, either to crop or to pad some pixels (or a combination of the two), in order to fit the image within a rectangle. The alpha parameter is a value between 0 and 1, sets this preference. With alpha=1 we will retain all pixels and only pad. With alpha=0 there is no padding, only cropping.
In the example, both the new and the old image are w by h pixels. See the reference manual for more details.

Exercise 2. Calibrate your own camera

Write a python script to calibrate your own camera.

  1. The tutorial in the first week demonstrated how to open your camera with OpenCV and capture images.
  2. You can find a chessboard pattern by searching the web, and print it.
  3. The tutorial above shows how to perform the calibration.
  4. Note the hint in the introduction to the tutorial. You can loop until you are able to pick up the chessboard pattern in the camera view.

You have some leeway in how you solve this problem. If you want to capture the images first, and then calibrate off-line, that is ok. If you want to calibrate with a live camera, that is also ok. Feel free to experiment, and learn what you want to learn.

Remark When I attempted this exercise, OpenCV only picked up the chessboard pattern in a fraction of the images. Hence, the recommendation in the tutorial is sound. Capture images in a loop and check for corners straight away. Keep moving the chessboard until you have enough images with detectable corners.

Remark 2 We have a couple of cameras and chessboards to use in class. Feel free to test multiple cameras.

Remark 3 We had some difficulty opening an external camera. It seems that cv2.VideoCapture takes a second argument which may be critical. The following example shows the idea.

import cv2
# define a video capture object
# try different number for the first argument if 1 doesn't work. 0,1,2,3 mostly
vid = cv2.VideoCapture(1,cv2.CAP_DSHOW) 


while (vid.isOpened()):
    # Capture the video frame by frame
    ret, frame = vid.read()
    # Display the resulting frame
    cv2.imshow('frame', frame)

    # the 'q' button is set as the quitting button you may use any
    # desired button of your choice
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# After the loop release the cap object
vid.release()
# Destroy all the windows
cv2.destroyAllWindows()

Exercise 3.

What makes a distorted image?

Try a range of angles and other condition to find out when straight lines become curved.

Test your calibration from Exercise 2 on the worst images you can make. You do not have to repeat the calibration exewrcise; just use the calibration result to undistort new images. Does it work adequately?

Exercise 4. (Optional)

  • Exercise 3.3 in Ma (2004) page 62. (Obviously, you can use Python or any other language instead of Matlab.)

Debriefing

  • Questions and Answers

Notes

  • Non-linear calibration is not a matrix operation
  • It is not an operation on the luminance values of the pixels

  • Instead

    1. The pixels are moved around in the plane, off the orignal grid
    2. Then we need to resample the image to get a new set of pixel in the original grid
  • OpenCV does not detect corners in all images and the tutorial recommends running a loop searching for corners while you move the chessboard

  • You may have to tweak the delays so that you see what you are doing in real time. I had to fiddle a lot to get this right.

Additional Material