Beskjeder
This Monday, 02 November, we will have a reading group. Here is the schedule:
12:15 - 12:45: TNN group presents "Learning to summarize with human feedback" (including 10 minutes Q&A)
12:45 - 13:00: Break
13:00 - 13:30: MPLT group presents "Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback" (including 10 minutes Q&A)
13:30 - 14:00: Miscellaneous, if need be
Important: everyone must read all the papers to be prepared for the discussion. The presenting team will walk through the most important aspects of each paper, probably identifying parts that may be debatable, counter-intuitive, surprising, unclear, etc. The presenting te...
Next week, we will have individual per-team consultations with the internship advisors, scheduled for two days: Monday, October 5 and Wednesday, October 7. The consultations are aimed at discussing the milestone progress of your team and its training roadmap.
Per-team schedule
Monday, October 5 (Shell):
- 12:15 - 12:37 NordLLM
- 12:38 - 13:00 MPLT
- 13:15 - 13:37 TNN
- 13:38 - 14:00 Spark...
Next Monday, 28 September, we will have a reading group. Here is the schedule:
12:15 - 12:45 BigFamLLM team presents "An empirical analysis of compute-optimal large language model training" (2022)
12:45 - 13:30 SM: Sjølv-merksemd team presents "Scaling Data-Constrained Language Models" (2023); with a break
13:30 - 14:00 ChiGerNo team presents "Small-Scale Experiments: Are We There Yet?" (2026)
Important: everyone must read all the three papers to be prepared for the discussion. The presenting team will walk through the most important aspects of each paper, probably iden...
Unfortunately, we had to skip one hackathon (on September 9).
Because of this, the deadline to submit Deliverable 2 (Training roadmap) is now extended to October 2.
The detailed description of how the deliverable should look like will be published by September 21st at the latest.
Next Monday, 14 September, we will have a reading group, not a regular lecture. Here is the schedule:
- 12:15 - 12:45 NordLLM team presents the FineWeb paper
- 12:45 - 13:30 Spark team presents the FineWeb-2 paper (with a break)
- 13:30 - 14:00 BeLLM team presents the HPLT-3 paper
Important: everyone must read all the three papers to be prepared for the discussion. The presenting team will walk through the most ...
This is the week to form developer teams. each team should comprise at least three (and at most five) interns, ideally with complementary skills and compatible chemistry. You will be working closely within your team throughout the full internship duration. Team work is obligabory in IN5640 (except for special individual circumstances, in which case we ask that you make contact with us at in5640-help@ifi.uio.no).
To support team formation, we will share with the internship group the individual resumés we have received (but not the more personal motivation letters); to limit visibility to IN5640 participants, we will post the files on our Mattermost channel, tomo...
We have just emailed instructions for gaining access to the national Olivia supercomputer to everyone who has submitted the first deliverable. If you have not received email from us today but consider yourself an active participant in IN5640, please make contact with us at in5640-help@ifi.uio.no. If you will not be taking the course, after all, please unregister in StudentWeb.
This coming week, we will proceed with the formation of developer teams, each comprising 3–5 interns. Account creation on Olivia may take a day or two, and everyone will need access for our first hackathon this coming Wednesday (14:15–16:00 in Datastue Limbo). Thus, please make sure to follow the instructions in our email and submit your Olivia account request no later than Monday evening, August 24.
IN5640 is currently overbooked. Quite a few people signed up in StudentWeb today, even though they did not attend the kick-off meeting yesterday. We are of course happy about the great interest, but we need to keep the group of participants to a manageable number due to limited compute available to us this fall.
We will try to squeeze in as many people as possible, but we cannot confirm finally before early Monday next week. Please watch this space carefully. In the meantime, please make sure to submit all mandatory steps this week, viz. (a) the on-line questionnaire from the kick-off meeting and (b) the first individual deliverabl...
We emphasize good, continuous work habits. The first deliverable this fall is an individual exercise and needs to be submitted no later than Friday, August 21, 2026.
We are not recording lectures this year, but we will typically publish slide copies of presentations. However, we emphasize direct contact and interactive discussions, hence will try our utmost to encourage regular attendance in both the Monday (“all-hands”) and Wednesday (“hackthon”) slots.
The course will kick-off on Monday, August 17, 12:15–14:00 in seminar room Shell. Note that in this course, we adopt the metaphor of a four-month, part-time internship in a leading LLM development laboratory. Prospective participants are expected to attend the kick-off session in person.
Meanwhile, we will prepare for the formation of developer teams of 3–5 interns (or “students”) and for access to the national supercomputer Olivia, where we will enjoy GPU compute capacity worth hundreds of thousands of Norwegian kroner. Students who cannot attend on August 17, must ...