Revision e6153c421b37f5b158c102b87e94cdee0327fbbe (click the page title to view the current version)
Changes from e6153c421b37f5b158c102b87e94cdee0327fbbe to 3a3a24df8554252491842ff0328fedeb00658bd3
---
title: Camera Calibration
categories: session
---
# Briefing
**Under construction** Links may not yet work.
The **goal** today is to understand what camera calibration is, why
we need it, and how to do it in OpenCV (python).
The **exercise** is a standard tutorial, which should be straight forward.
To understand what calibration *is*, we need first to note that there are
two kinds. It is easy to get confused, because many authors consider only
one kind, depending on context, without making it explicit.
I have prepared two talks to explain this:
+ [Linear Calibration](http://www.hg.schaathun.net/talks/maskinsyn2023/calibration/)
assumes ideal projection and aims to determine all
the details of the mapping from 3D to 2D.
+ [Non-Linear Distortion](http://www.hg.schaathun.net/talks/maskinsyn2023/distortion/)
is the result of imperfect lenses which make the
camera model inaccurate. The projection is no longer ideal.
Lecture Notes on [Camera Mathematics]() from previous years are still available,
but I think the talks above are easier to understand.
# Exercises
We are going to do the calibration in practice.
**Note that ...**
+ OpenCV does not detect corners in all images and the tutorial recommends
running a loop searching for corners while you move the chessboard
+ [Sample Code](calibration.py)
+ You may have to tweak the delays so that you see what you are doing
in real time. I had to fiddle a lot to get this right.
## Exercise 1. Tutorial
1. Complete the tutorial:
[Calibration in OpenCV](https://docs.opencv.org/master/dc/dbb/tutorial_py_calibration.html)
+ [Test Images](https://github.com/ChihaoZhang/camera-calibration-and-image-undistortion/tree/master/left)
2. Discuss the following questions, using the hints below and the
OpenCV documentation.
+ Where in this tutorial do you calculate the camera intrinsic matrix?
+ How can you find the radial distortion parameters?
+ What can you say about the other variables calculated in the
tutorial?
The most critical calibration functions are the following.
```python
retval, cameraMatrix, distCoeffs, rvecs, tvecs =
cv2.calibrateCamera(objectPoints, imagePoints, imageSize)
```
The output parameters are explained in the
[reference documentation](https://docs.opencv.org/master/d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d).
However, the documentation is written for the C++ syntax.
Hence, most of the output parameters will appear as input parameters
in the documentation. In the code example above, I have used the
same parameter names as in the C++ documentation.
```python
newcameramtx, roi = cv.getOptimalNewCameraMatrix(mtx, dist, (w,h), alpha, (w,h))
```
When the image is transformed to remove distortion, the result
is not rectangular. One has to choose, either to crop or to pad some
pixels (or a combination of the two), in order to fit the image within
a rectangle.
The `alpha` parameter is a value between 0 and 1, sets this preference.
With `alpha=1` we will retain all pixels and only pad.
With `alpha=0` there is no padding, only cropping.
In the example, both the new and the old image are `w` by `h` pixels.
See the [reference manual](https://docs.opencv.org/master/d9/d0c/group__calib3d.html#ga7a6c4e032c97f03ba747966e6ad862b1) for more details.
## Exercise 2. Calibrate your own camera
Write a python script to calibrate your own camera.
1. The tutorial in the first week demonstrated how to open your camera
with OpenCV and capture images.
2. You can find a chessboard pattern by searching the web, and print it.
3. The tutorial above shows how to perform the calibration.
4. Note the hint in the introduction to the tutorial.
You can loop until you are able to pick up the chessboard pattern
in the camera view.
You have some leeway in how you solve this problem.
If you want to capture the images first, and then calibrate off-line,
that is ok.
If you want to calibrate with a live camera, that is also ok.
Feel free to experiment, and learn what you want to learn.
**Remark**
When I attempted this exercise, OpenCV only picked up the chessboard
pattern in a fraction of the images. Hence, the recommendation in the
tutorial is sound.
Capture images in a loop and check for corners straight away.
Keep moving the chessboard until you have enough images with detectable
corners.
**Remark 2**
We have a couple of cameras and chessboards to use in class.
Feel free to test multiple cameras.
**Remark 3**
We had some difficulty opening an external camera.
It seems that `cv2.VideoCapture` takes a second argument which may be
critical. The following example shows the idea.
```pythone
import cv2
# define a video capture object
# try different number for the first argument if 1 doesn't work. 0,1,2,3 mostly
vid = cv2.VideoCapture(1,cv2.CAP_DSHOW)
while (vid.isOpened()):
# Capture the video frame by frame
ret, frame = vid.read()
# Display the resulting frame
cv2.imshow('frame', frame)
# the 'q' button is set as the quitting button you may use any
# desired button of your choice
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# After the loop release the cap object
vid.release()
# Destroy all the windows
cv2.destroyAllWindows()
```
## Exercise 3.
What makes a distorted image?
Try a range of angles and other condition to find out when straight lines
become curved.
Test your calibration from Exercise 2 on the *worst* images you can make.
## Exercise 4. (Optional)
+ Exercise 3.3 in Ma (2004) page 62.
(Obviously, you can use Python or any other language instead of Matlab.)
# Debriefing
+ Questions and Answers
# Notes
+ Calibration is not a straight-forward matrix operation
+ Non-linear calibration is not a matrix operation
+ It is not an operation on the luminance values of the pixels
+ Instead
1. The pixels are moved around in the plane, off the orignal grid
2. Then we need to resample the image to get a new set of pixel
in the original grid
+ OpenCV does not detect corners in all images and the tutorial recommends
running a loop searching for corners while you move the chessboard
+ [Sample Code](calibration.py)
+ You may have to tweak the delays so that you see what you are doing
in real time. I had to fiddle a lot to get this right.
# Additional Material
+ Other tutorials:
[Camera Calibration and 3D Reconstruction](https://docs.opencv.org/master/d9/db7/tutorial_py_table_of_contents_calib3d.html)
