Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Bias-Variance Tradeoff

Computer Science Dept, Ferdowsi University of Mashhad

<Figure size 1200x1000 with 12 Axes>
<Figure size 500x300 with 1 Axes>
Notebook Cell

Exercises

  1. Starting from y=f(x)+εy=f(x)+\varepsilon with E[ε]=0\mathbb{E}[\varepsilon]=0 and Var(ε)=σ2\mathrm{Var}(\varepsilon)=\sigma^2, show that E[(y−f^(x))2]=σ2+(E[f^(x)]−f(x))2+Var(f^(x))\mathbb{E}\big[(y-\hat f(x))^2\big] = \sigma^2 + \big(\mathbb{E}[\hat f(x)]-f(x)\big)^2 + \mathrm{Var}\big(\hat f(x)\big). State precisely where each assumption is used.

  2. Fit polynomials of degree 1,…,121,\dots,12 to a noisy sine of your choice and plot training and test error on one axis. Mark the degree at which the two curves separate and relate it to the three terms above.

  3. The number of parameters is not a reliable measure of flexibility: give one example from the notebook where a model with fewer parameters has larger variance, and explain the mechanism.