# Neural Networks

## Changes from 444e5a3daf141740f8beedb68560a6eaf9c0fd3f to current

---
title: Neural Networks
categories: session
---

+ [Briefing 10 November](ANN) - Principles of training and testing
+ [Briefing 11 November](CNN) - Convolutional Networks - what are they?

+ [PyTorch Quickstart](https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html)
+ the quickstart tutorial is part of [Learn the Basics](https://pytorch.org/tutorials/beginner/basics/intro.html)
+ [Learn the Basics](https://pytorch.org/tutorials/beginner/basics/intro.html)
+ Szeliski 2022 Chapter 5

[Briefing](ANN)

# Exercise 1. Basic tutorial.

I have added a couple of exercises to the official
[PyTorch Quickstart](https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html).

+ Please reflect upon and discuss the questions.

# Exercise 3. Regression.

One in-house project is to classify images of remote galaxies,
which are often distorted due to the gravity of dark matter.
This effect is known as gravitational lensing.
# Exercise 2. Convolutional Neural Networks

A sample dataset can be found at
[github](https://github.com/CosmoAI-AES/datasets2022/Exercise2022).
This directory contains
1. Complete the [CIFAR-10 Tutorial](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)
2. Compare the approach to Exercise 1.  What is different?
What is the same?

# Exercise 3. Evaluation and Analysis

+ A data file, sphere-pm.csv
+ 10000 images of distorted galaxies
+ A python file Dataset.py defining a subclass of Dataset
to manage these data
1.  Copy the code out of the tutorials so that you can run it
in your regular environment (IDE, command line, etc.).
2.  Test one of the networks with different numbers of epochs.
Record the accuracy both on the training set and on the testing
set for each epoch.
3.  Plot the training and testing accuracy as a function of the number
of epochs.  What do you see?
4.  What is the ideal number of epochs?
5.  Calculate a confidence interval for the accuracy at the ideal
number of epochs.  What do you think of the quality of the network?
What do you think about the quality of the assessment of the network?

The CSV file has the form

index,filename,source,lens,chi,x,y,einsteinR,sigma,sigma2,theta,nterms
"00001",image-00001.png,s,p,50,30,40,19,31,0,0,16

The interesting columns are the filename which points to the
input image,  and
the four output variables $x$, $y$, einsteinR, and $\sigma$.
The other columns are associated with more advanced problem instances
and should be ignored.
Note that you do not have to rerun the entire process for more epochs.
You can test the network after each epoch and continue training.

1.  Study the Dataset class Dataset.py.  How is the dataset managed?
2.  Use this class to test if you can train a network to determine the
four outputs for an image.
If you do not see the effect of *overtraining*, try to use smaller training sets and see how it changes the behaviour of the network.

# Exercise 4.  Managing Data.
# Exercise 4.  Different networks (optional)

Doing a tutorial is good, but it is little use if you
you can only use the sample data.
The goal of this exercise is to learn to manage other
datasets.
Try to change the neural network in Exercise 1 and 2.
Can you improve the performance?

Please be aware that deep learning is usually extremely compute
intensive, and you can easily run into a problem which takes days
to compute.  The immediate solution to this is to
# Exercise 5.  More examples (optional)