2 Tutorial 4.2: Linear and non-linear classifiers
Reading: Stephen Marsland: Chapter 3.4-3.5 and the introduction for Chapter 4.
2.1 Problem 1: A Neural Network for the XOR problem
2.1.1 Step 1: A new module
- 1.
- Review your solution to Problem 2 of Tutorial 3.2.
- 2.
- Create a new module
ANN
, where you import your existingPerceptron
module.import Perceptron
- 3.
- Define a data type
Network
which consist of several layers. Add it to theANN
module.
2.1.2 Step 2: A network for XOR
Recall the XOR classification problem in Figure 1a. We can use the solution (Figure 1b) to
test our neural network and the recall
function before we have to start thinking about
training.
- 1.
- Define a neural network
xorNetwork
with the weights given in Figure 1b. Use theNetwork
type which we defined above. - 2.
- Show the network in GHCi, to check that you have not made a syntax error or similar.
xorNetwork
- 3.
- Now we need to implement a recall function. Add the type declaration to your
ANN
module.recallNetwork :: Network −> [Double] −> [Double]
- 4.
- Implement
recallNetwork
, usingrecallLayer
from Problem 2 of Tutorial 5. - 5.
- Discuss: Which is the output layer? Is it first or last element in the list which makes up the
Network
? - 6.
- Finally, test your definitions with the following evaluations:
recallNetwork xorNetwork [0,0]
recallNetwork xorNetwork [0,1]
recallNetwork xorNetwork [1,0]
recallNetwork xorNetwork [1,1] - 7.
- Discuss: Is the output as expected?
2.1.3 Step 3: Bug search
There are two common sources of errors in this network/implementation.
- 1.
- What is the value of the threshold function at 0? I.e. does the neuron fire when the sum is exactly 0. In floating point problems, this hardly matters, but with the binary xor problem it does. If your test fails, try to change the threshold function.
- 2.
- What is the sign of the quasi-input ? Very often we use -1, but some authors use +1.