Revision b6398d965879a8c814f6b5561f81169f62dfccc8 (click the page title to view the current version)

ANN

Changes from b6398d965879a8c814f6b5561f81169f62dfccc8 to 0c624e8ca81953eb04087720851e3c22f8417c53

---
title: Neural Networks Lecture
categories: lecture
---

# Briefing

## What is a newural network

+ The single Neuron
    + Weighted Input
    + Activation
+ The network model
    + Input/Output
    + Weights
    + Activation Function
+ The Tensor Model

## Output and Loss Function

+ Classification versus Regression

**MSE**

$$L = (x-y)^2$$

**CrossEntropy**

$$L = \log \frac{ \exp x_{y} } { \sum \exp x_i }$$

## Training

+ Optimisation problem
    + tune the weights to minimise the loss function
    + if the activation function is differentiable, the entire system is
    + different optimisation algorithms;
      trust the API or do a more advanced module

## Activation Functions

+ Threshold functions
+ Approximations to the threshold function
+ Logistic: $f(x) = \frac1{1+e^{-\beta x}}$
+ ReLU: $f(x)=\max(x,0)$
    - not differentiable


## Sample Problem