Convolutional Neural Networks Lecture
Briefing
What is a Convolutional Network
- Convolutional Layers
- \(3\times3\) or \(5\times5\) (possibly \(7\times7\))
- Pooling
- Often maximum; sometimes average or other functions
- Convolution-Detector-Pooling = Convolutional Unit
- Convolution
- Activation (Detector)
- Pooling
Why is convolution suitable for images?
Features of CNN
- Capacity (degrees of freedom)
- fewer weights (DOF) per layer
- Representation learning
- may learn edge or corner detection for instance
- Location invariance
- Hierarchies; serial and parallel modularisation
- Fully connected layer at the end.
- convolutional units learn features
- last layer uses the features like a traditional ANN with one hidden layer
Considerations in CNN
- Padding (edge conditions for convolution)
- Batch Normalisation
- mini batches
- Filter/Kernel
Other important features (any ANN)
- Normalisation
- Over- and under-training
- Plot over epochs
- Feature detection
Sample Network
class ConvNet(nn.Module):
def __init__(self, num_classes=10):
super(ConvNet, self).__init__()
self.conv_unit_1 = nn.Sequential(
nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(16),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2))
self.conv_unit_2 = nn.Sequential(
nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(32),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2,stride=2))
self.fc1 = nn.Linear(7*7*32, 128)
self.fc2 = nn.Linear(128, 10)
def forward(self, x):
out = self.conv_unit_1(x)
out = self.conv_unit_2(out)
out = out.view(out.size(0), -1)
out = self.fc1(out)
out = self.fc2(out)
out = F.log_softmax(out,dim=1)
return out