Revision beba4bfdebe5f7d950f6c78257abc981754f56e1 (click the page title to view the current version)


Changes from beba4bfdebe5f7d950f6c78257abc981754f56e1 to 66aa985a4fdd9639cbc3dc226df51d8b765cbece

title: Evaluation and Statistics

## Heuristics

+ Accuracy
+ FP/FN rate
+ Confusion Matrix

# Exercise 1

Review the machine learning systems you studied last week.
Calculate the confusion matrix for each of the systems.

+ Are the errors reasonably balanced?
+ Are any classes particularly difficult to detect?

# Exercise 2

Reviewing again each of the systems from last week.

1. Calculate the false positive and false negative rates.
2. Estimate the standard deviation of the error rates.
3. Assess the quality of the machine learning system.
   Can you be confident that the error probability is satisfactory?
4. How large does the test set have to be to make a confident assessment? 

# Debrief

+ Demo: [python module/script](Python/