## Revision 66aa985a4fdd9639cbc3dc226df51d8b765cbece (click the page title to view the current version)

# Evaluation and Statistics

## Heuristics

- Accuracy
- FP/FN rate
- Confusion Matrix

# Exercise 1

Review the machine learning systems you studied last week. Calculate the confusion matrix for each of the systems.

- Are the errors reasonably balanced?
- Are any classes particularly difficult to detect?

# Exercise 2

Reviewing again each of the systems from last week.

- Calculate the false positive and false negative rates.
- Estimate the standard deviation of the error rates.
- Assess the quality of the machine learning system. Can you be confident that the error probability is satisfactory?
- How large does the test set have to be to make a confident assessment?

# Debrief

- Demo: python module/script