Plans for week 45, note change in deadline for project 2
Dear all, we hope your weekend has evolved in the best possible way. An important thing first:
We changed the deadline for project 2 till Monday November 9 (midnight) and you obviously have an additional day (Tuesday Nov 10) for corrections. Hope this helps.
On Tuesday this, since we don't have exercises, we will just work on project 2, see the slides for Tuesday, there are some slides which may be useful for your analysis part. Else, we are basically done with the feedback for project 1 and will upload these to canvas tomorrow (Monday and Tuesday). Hopefully, this feedback may be of use for project 2.
Last week we discussed convolutional neural networks, essentially the method for image recognition and more. On Monday we will start the lecture with a reminder and a short discussion on how to use Tensorflow with Keras or PyTorch for setting up a CNN. After that, we will discuss the last eep learning method this semester, namely recurrent neural networks (RNN). We will end that part the first hour next week and then for the rest of the semester we will focus on mathematically much simpler methods, namely decision trees and the family of ensemble methods (random forests, bagging, gradient boosting and other methods).
We will come back to alternatives for project 3 during the Tuesday session next week. There we will present project 3 and ask you to propose alternative datasets as well.
The solutions to the weekly exercises of week 43 and 44 are available as the files exercisesweek*_solution.ipynb in the Weekly material folders week43 and week44. And you should also have the solutions for weeks 41 and 42.
This week the plan is thus as follows:
Plan for week 45
Monday lecture (2×45 min):
1. Recap of convolutional networks (15 min): what week 44 established, and the Keras and PyTorch programs for MNIST.
2. From feed-forward to recurrent networks
3. The architecture: the simple RNN, folded and unrolled; the RNN as a dynamical system; where the recurrence comes from.
4. The layers in Keras and PyTorch: what SimpleRNN and nn.RNN compute, line by line.
5. Backpropagation through time: the recursion, and what loss.backward() does with it.
6. Why long sequences are hard: vanishing and exploding gradients, measured with autograd; clipping; a network that learns the rule of a damped oscillator.
7. Three applications: a sine wave, MNIST row by row, and the adding problem where the simple RNN runs out of memory.
Next Monday, first hour: gated cells (LSTM, GRU) and sound. Then decision trees and start ensemble methods.
Tuesday: work on Project 2 (deadline Monday November 9); no weekly exercises this week. Take a look at the slides for Tuesday and see if the questions there are relevant for your own analysis
Reading: Chapter 11, Sections 11.111.6 and 11.8; Chapter 10 for the recap. The code is in week45monday.ipynb.
The material for week 45 is at https://github.com/EducationalMaterialUiO/MachineLearningUiO/tree/main/doc/WeeklyMaterial/week45
Best wishes to you all and good luck with finalizing project 2.
Amanda, Amund, Britt, Kim, Morten, and Viktor