Plans for week 36, Aug 31-September 4

Dear all, welcome back to FYS-STK.  We hope you had a wonderful weekend.  This week we will continue our discussion of the train-test data we discussed last week, but now by introducing the so-called Bias-Variance tradeoff and some statistics. Our plans are

 A reminder of OLS, Ridge and Lasso from last week, approx first 10-15 mins of Monday's lecture

 The training/test figure as function of polynomial degree; the question this whole week answers

  The statistical toolbox (Sections 2.3-2.8 of the book): expectation values, sample estimators, the central limit theorem, and the statistics of the OLS and Ridge estimators

 Deriving the bias-variance tradeoff, Eq. (2.47) of the book, with code

 Resampling methods: the bootstrap and k-fold cross-validation

Tuesday flipped classroom session: your questions, then two cases that will keep us busy, namely the bias-variance decomposition by the bootstrap, and cross-validation for the Ridge penalty.  Bring your laptop.

Reading suggestions

Chapter 2, Sections 2.3-2.12 of the book, parts of chapter 2 of Goodfellow et al and  Hastie et al., Sections 7.1--7.5, 7.10 (cross-validation) and 7.11 (bootstrap).

Look forward to meet you all again. The weekly material is at https://github.com/EducationalMaterialUiO/MachineLearningUiO/tree/main/doc/WeeklyMaterial/week36. It contains the weekly exercises (https://github.com/EducationalMaterialUiO/MachineLearningUiO/blob/main/doc/WeeklyMaterial/week36/exercisesweek36.ipynb), the slides for the lecture Monday and the flipped class on Tuesday as well as the jupyter-notebook with codes discussed on Monday and Tuesday, see https://github.com/EducationalMaterialUiO/MachineLearningUiO/blob/main/doc/WeeklyMaterial/week36/week36tuesday.ipynb. 

Best wishes to you all

Amanda, Amund, Brit, Kim, Morten, and Viktor