Plans for week 39, September 21-25

Dear all, we hope you are having a great weekend. 

The plans for this coming week are 

Monday lecture (2×45 min):

▶ First 20 min: stochastic gradient descent once more the minibatch gradient as an estimate, what an epoch costs, the example where SGD beats gradient descent (Section 4.7; Project 1 part h)

▶ Logistic regression: why a straight line cannot classify, the logistic function and the log-odds, the likelihood and the cross entropy, gradient, Hessian and convexity, gradient descent against Newton, separability and penalties, the softmax (Sections 5.15.7)

▶ SGD and the adaptive methods on a logistic regression with 50 000 observations, and what Adam can and cannot do about features that are not standardised (Section 5.9)

▶ Last part: logistic regression drawn as a single neuron the simple perceptron and its learning rule (Section 5.8.1): the stepping stone to neural networks. The same picture read backwards, automatic differentiation (Section 5.8.2), waits until week 40

Tuesday (flipped classroom): your questions, then how to write the report abstract, introduction, conclusion, references and the declaration of LLM use (weekly exercises; week39tuesday.ipynb).

Reading: Chapter 5, Sections 5.15.9; Section 4.7 once more. Hastie et al. Section 4.4; Goodfellow et

al. Sections 5.7.1 and 8.18.3. Deadline for the weekly exercises: Friday September 25 at midnight.

The material for this week is at https://github.com/EducationalMaterialUiO/MachineLearningUiO/tree/main/doc/WeeklyMaterial/week39

Keep in mind that we also update other parts of the teaching material on a weekly basis. For example, the lecture notes get typically revised during weekends. 

Finally, we would like to apologize for an extremely long exercise set this last week. We (=Morten)  got carried away and din't realize before it was too late that the exercise set was way too long. The hope is however, since everything you do with the exercises can be transferred t the project, that what you did turns into some useful for your final project. 

This week's exercises, as you may see from the the file exercisesweek39.ipynb, focus only on writing parts of the project, the abstract, intro, conclusions and perspectives (plus an LLM declaration). We will try to give you feedback on these exercises before the deadline, so that you can improve further these parts of the project.

And never hesitate to ask questions.

Best wishes to you all,

Amanda, Amund, Brit, Kim, Morten, and Viktor