## Revision 8e8b5e1b7f285bf347faa6ade2d5acd6e72e63ae (click the page title to view the current version)

# Introduction

## Changes from 8e8b5e1b7f285bf347faa6ade2d5acd6e72e63ae to ca74468973458085c1b0ec3276774908662fd9f9

--- title: Introductory Session to Machine Learing categories: session 3D --- + [Up: [Overview]()]-[Next: [3D Modelling]()] # Reading + Ma 2004 Chapter 1. + (Szeliski 2022 Chapter 1) # Session 1. **Briefing** [Introduction](http://www.hg.schaathun.net/talks/maskinsyn/intro.html) 2. **Exercise** [Python Setup](#practical) (below) 3. **Debrief** + Recap on linear algebra (introductory slides continued) + Lecture Notes on [Change of Basis]() # Practical: Python Setup {#practical} Today's task is to install and test a number of software packages on your computers. The goal is not to produce the output that I ask for, but to make sure the software works for *you* on *your* system. ## Install Python We will use Python 3 in this module. There is a myriad of different ways to install and use python, and you have probably used one or another in other modules already. During this module, we will occasionally work with external USB cameras, and we will also use libraries which open windows to view video and graphics. This can be difficult to get to work in Jupyter, but apart from this, you should use whatever platform you are most comfortable with. ## Alternative plaforms ### Working with the command line You need to install the following. 1. [Python](https://www.python.org/downloads/) 2. [pip](https://packaging.python.org/tutorials/installing-packages/). (the package manager for python) 3. [iPython](https://ipython.org/install.html) (a more convenient interactive interpreter) How you install these three packages depends on your OS. In most distroes you can use the package system to install all of this, for instance in Debian/Ubuntu: ```sh sudo apt-get install ipython3 python3-pip ``` (Python is installed automatically as a dependency of iPython3. Note, you have to specify version 3 in the package names, lest you get Python 2.) ### Anaconda/Miniconda Anaconda and Miniconda is a platform for managing python installations. This is recommended for Windows users. For installation, [download miniconda](https://docs.conda.io/en/latest/miniconda.html) or [anaconda](https://www.anaconda.com/products/individual) + [Miniconda — Conda documentation](https://docs.conda.io/en/latest/miniconda.html) Miniconda¶. Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages, including pip, zlib and a few others. + docs.conda.io Create virtualenvironment with ```sh conda create -n <name> python=<version> ``` e.g. ```sh conda create -n maskinsyn python=3.8 ``` Activate with ```sh conda activate <name> ``` e.g. ```sh conda activate maskinsyn ``` Install with `pip install`, or `conda install` [Creating environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands) [Managing environments — conda 4.10.3.post24+d808108b6 documentation](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands) An explicit spec file is not usually cross platform, and therefore has a comment at the top such as # platform: osx-64 showing the platform where it was created. This platform is the one where this spec file is known to work. ## Install Python Packages Python packages are installed most easily using python's own packaging tool, pip, which is independent of the OS. It is run from the command line. Depending on how you installed pip, it may be a good idea to upgrade ```sh pip3 install --upgrade pip ``` Then we install the libraries we need. You can choose to install either in user space or as root. User space: ```sh pip3 install --user matplotlib numpy opencv-python ``` As root: ```sh sudo pip3 install matplotlib numpy opencv-python ``` + numpy is a standard library for numeric computations. In particular it provides a data model for matrices with the appropriate arithmetic functions. + matplotlib is a comprehensive library for plotting, both in 2D and 3D. + [OpenCV](https://opencv.org/) is a Computer Vision library, written in C++ with bindings for several different languages. A third installation alternative is to use [Virtual Environments](https://docs.python.org/3/tutorial/venv.html), which allows you to manage python versions and dependencies separately for each project. This may be a good idea if you have many python projects, but if this is your first one, it is not worth the hassle. ## Run iPython Exactly how you run iPython may depend on you OS. In Unix-like systems we can run it straight from the command line: ```sh ipython3 ``` This should look something like this: ``` georg$ ipython3 Python 3.7.3 (default, Jul 25 2020, 13:03:44) Type 'copyright', 'credits' or 'license' for more information IPython 7.3.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: print("Hello World") Hello World In [2]: import numpy as np In [3]: np.sin(np.pi) Out[3]: 1.2246467991473532e-16 In [4]: np.sqrt(2) Out[4]: 1.4142135623730951 In [5]: ``` ## Some 3D Operations In this chapter, we will define a simple 3D Object and display it in python. The 3D object is an irregular tetrahedron, which has four corners and four faces. ```python import numpy as np from matplotlib import pyplot as plt from mpl_toolkits.mplot3d.art3d import Poly3DCollection ``` Firtsly, we define the three corners of the tetrahedron. ```python corners = [ [-1,0.5,0.5], [+1,0.5,0.5], [0,-0.5,0.5], [0,0.5,-0.5] ] ``` Each face is adjacent to three out of the four corners, and can also be defined by these corners. ```python face1 = [ corners[0], corners[1], corners[2] ] face2 = [ corners[0], corners[1], corners[3] ] face3 = [ corners[0], corners[2], corners[3] ] face4 = [ corners[1], corners[2], corners[3] ] ``` To represent the 3D structure for use in 3D libraries, we juxtapose all the faces and cast it as a matrix. ```python vertices = np.array(face1+face2+face3+face4,dtype=float) print(vertices) ``` Observe that the vertices (corners) are rows of the matrix. The mathematical textbook model has the corners as columns, and this is something we will have to deal with later. We define the 3D object `ob` as follows. ```python ob = Poly3DCollection(vertices, linewidths=1, alpha=0.2) ``` The `alpha` parameter makes the object opaque. You may also want to play with colours: ```python ob.set_facecolor( [0.5, 0.5, 1] ) ob.set_edgecolor([0,0,0]) ``` To display the object, we need to create a figure with axes. ```python plt.ion() fig = plt.figure() ax = fig.add_subplot(111, projection='3d') plt.show() ``` Note the `plt.ion()` line. You do not use this in scripts, but in `ipython` it means that control returns to the prompt once the figure is shown. It is necessary to continue modifying the plot after it has been created. Now, we can add our object to the plot. ```python ax.add_collection3d(ob) ``` **Note** It seems that some versions of matplotlib do not work with the above definition of `vertices`, and rather require a 3D array, as follows: ```python vertices = np.array([face1,face2,face3,face4],dtype=float) print(vertices) ``` You can try that, but you may run into trouble when we rotate the figure later. I am looking further into the compatibility between versions. Quite likely, the object shows outside the range of the axes. We can fix this as follows: ```python s = [-2,-2,-2,2,2,2] ax.auto_scale_xyz(s,s,s) ``` These commands make sure that the axes are scalled so that the two points `(-2,-2,-2)` and `(2,2,2)` (defined in the list `s`) are shown within the domain. ## Rotation and Translation of 3D objects Continuing on the previous section, our 3D object `ob` is defined by the `vertices` matrix, where all the rows are points in space. Motion is described by matrix operations on `vertices`. ### Translation Let us define another vector in $\mathbb{R}^3$ and add it to each point. ```python translation = np.array( [ 1, 0, 0 ], dtype=float ) v2 = vertices + translation print(v2) ``` Not that this operation does not make sense in conventional mathematics. We have just added a $1\times3$ matrix to an $N\times3$ matrix. How does python interpret this in terms of matrices? To see what this means visually in 3D space, we can generate a new 3D object from `v2`. We use a different face colour for clarity. ```python ob2 = Poly3DCollection(v2, linewidths=1, alpha=0.2) ob2.set_facecolor( [0.5, 1, 0.5] ) ob2.set_edgecolor([0,0,0]) ax.add_collection3d(ob2) ``` How does the new object relate to the first one in 3D space? **TODO** Check ### Rotation In the previous test, we added a vector to the nodes in the 3D polyhedron. Let's try instead to multiply by a matrix, like this: ```python theta = np.pi/6 R = np.array( [ [ np.cos(theta), -np.sin(theta), 0 ], [ np.sin(theta), np.cos(theta), 0 ], [ 0, 0, 1 ] ], dtype=float ) v3 = np.matmul(vertices,R) print(v3) ``` This gives us a new polyhedron, like this: ```python ob3 = Poly3DCollection(v3, linewidths=1, alpha=0.2) ob3.set_facecolor( [1, 0.5, 0.5] ) ob3.set_edgecolor([0,0,0]) ax.add_collection3d(ob3) ``` **TODO** Check ### Removing a 3D Object Unfortunately, `matplotlib.pyplot` is not designed for interactive construction or animation of 3D graphics, so some things are little bit tricky. However, it is possible to remove an existing object from the plot. Assume we still have the objects from the last few sections. First of all, let's look at the objects we have plotted: ``` In [12]: ax.collections Out[12]: [<mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x7fdde8a074f0>, <mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x7fdde89fe400>, <mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x7fddf39b28b0>] In [13]: ``` Here, `ax` is the axes system of the figure, and `ax.collections` is a collection of all of the objects that we have plotted. We can quite simply delete one and refresh the figure. ```python del ax.collections[1] plt.show() ``` What happens? ## Some Camera Operations ```python import numpy as np import cv2 as cv cap = cv.VideoCapture(0) ret, frame = cap.read() ``` Now, `ret` should be `True`, indicating that a frame has successfully been read. If it is `False`, the following will not work. ```python gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY) cv.imshow('frame', gray) cv.waitKey(1) ``` You should see a greyscale image from your camera. To close the camera and the window, we run the following. ```python cap.release() cv.destroyAllWindows() ``` This example is digested from the tutorial on [Getting Started with Videos](https://docs.opencv.org/master/dd/d43/tutorial_py_video_display.html). You may want to do the rest of the tutorial. ## Testing external cameras (optional) We have a box of external cameras for testing. It is useful to repeat the above exercise with an external camera. That means you have to change the camera selection (0) in this line: ``` cap = cv.VideoCapture(0) ``` In unix-like systems, you can use the device name, e.g. `/dev/video1` in lieu of the number 0. Thus you can double-check the camera connection with other capturing software. ## Showing a frame (optional) The following example may visualise rotations somewhat better than the tetrahedron given above. Try it, and observe to learn how it workse. **Step 1** Load libraries and set up a figure with axis system. ```python import numpy as np import matplotlib.pyplot as plt from matplotlib import pyplot as plt from mpl_toolkits.mplot3d.art3d import Poly3DCollection ax = plt.figure().add_subplot(projection='3d') ``` We can visualise just the unit vectors, using a quiver plot. We make the `qplot` function so that we can plot a rotated version afterwards. ```python e1 = np.array([1,0,0]) e2 = np.array([0,1,0]) e3 = np.array([0,0,1]) def qplot(e1,e2,e3,**kw): ax.quiver(0, 0, 0, *e1, colors="r",**kw ) ax.quiver(0, 0, 0, *e2, colors="g",**kw ) ax.quiver(0, 0, 0, *e3, colors="b",**kw ) qplot(e1,e2,e3) ``` If you want to, you can show the plot at this stage. Now we rotate the unit vectors, and make a new quiver plot. ```python R = np.array([ [ 0.1729, -0.1468, 0.9739], [ 0.9739, 0.1729, -0.1468], [ -0.1468, 0.9739, 0.1729] ]) qplot(np.matmul(R,e1),np.matmul(R,e2),np.matmul(R,e3)) ``` Finally, we need to scale and show the plot. The `set_xlim3d` and similar functions make arrow heads. ```python s = [-2,-2,-2,2,2,2] ax.auto_scale_xyz(s,s,s) ax.set_xlim3d([-2.0, 2.0]) ax.set_xlabel('X') ax.set_ylim3d([-2.0, 2.0]) ax.set_ylabel('Y') ax.set_zlim3d([-2, 2]) ax.set_zlabel('Z') plt.show() ``` ## STL Files (even more optional) If you have time, you can install and test the numpy-stl library as well. We shall use it [next week](3D Objects in Python#stl-files-and-the-stl-library). ## Other references - [Core Operations](https://docs.opencv.org/4.5.1/d7/d16/tutorial_py_table_of_contents_core.html) on images