Learning Foundations: Classification and Generalization

Status: placeholder — write lecture notes here before teaching.

Not in this course: SVM / kernels (other course), Bayesian methods and HMM (SML).

Goals¶

• Classification problem: train/test split

• kNN revisited as a nonparametric baseline

• Occam’s razor and why simple models can generalize

• Uniform convergence and the VC idea (intuition, not full theory)

Outline¶

1. Empirical risk vs true risk

2. Overfitting and model complexity

3. Why more data helps (informal)

4. Bias–variance link back to the bias–variance notebook

Sources¶

• Blum, Hopcroft, Kannan, Foundations of Data Science, Ch. 12 (selected)