Revision 0c624e8ca81953eb04087720851e3c22f8417c53 (click the page title to view the current version)
Neural Networks Lecture
Briefing
What is a newural network
- The single Neuron
- Weighted Input
- Activation
- The network model
- Input/Output
- Weights
- Activation Function
- The Tensor Model
Output and Loss Function
- Classification versus Regression
MSE
\[L = (x-y)^2\]
CrossEntropy
\[L = \log \frac{ \exp x_{y} } { \sum \exp x_i }\]
Training
- Optimisation problem
- tune the weights to minimise the loss function
- if the activation function is differentiable, the entire system is
- different optimisation algorithms; trust the API or do a more advanced module
Activation Functions
- Threshold functions
- Approximations to the threshold function
- Logistic: \(f(x) = \frac1{1+e^{-\beta x}}\)
- ReLU: \(f(x)=\max(x,0)\)
- not differentiable