---
title: Convolutional Neural Networks Lecture
categories: lecture
---

# Briefing

## What is a Convolutional Network

+ Convolutional Layers
    + $3\times3$ or $5\times5$ (possibly $7\times7$)
+ Pooling
    + Often maximum; sometimes average or other functions
+ Convolution-Detector-Pooling = Convolutional Unit
    + Convolution
    + Activation (Detector)
    + Pooling

Why is convolution suitable for images?

## Features of CNN

+ Capacity (degrees of freedom)
    + fewer weights (DOF) per layer
+ Representation learning
    - may learn edge or corner detection for instance
+ Location invariance
+ Hierarchies; serial and parallel modularisation
+ Fully connected layer at the end.
    - convolutional units learn features
    - last layer uses the features like a traditional ANN with one hidden layer

## Considerations in CNN

+ Padding (edge conditions for convolution)
+ Batch Normalisation
    + mini batches
+ Filter/Kernel

## Other important features (any ANN)

+ Normalisation
+ Over- and under-training
    + Plot over epochs
+ Feature detection

## Sample Network

```python
class ConvNet(nn.Module):
    def __init__(self, num_classes=10):
        super(ConvNet, self).__init__()

        self.conv_unit_1 = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=3, stride=1,  padding=1),
            nn.BatchNorm2d(16),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.conv_unit_2 = nn.Sequential(
            nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2,stride=2))
        self.fc1 = nn.Linear(7*7*32, 128)
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        out = self.conv_unit_1(x)
        out = self.conv_unit_2(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.fc2(out)
        out = F.log_softmax(out,dim=1)
        return out

```