28 September: Reading Group on Training Procedures

Next Monday, 28 September, we will have a reading group. Here is the schedule:

12:15 - 12:45 BigFamLLM team presents "An empirical analysis of compute-optimal large language model training" (2022)

12:45 - 13:30 SM: Sjølv-merksemd team presents "Scaling Data-Constrained Language Models" (2023); with a break

13:30 - 14:00 ChiGerNo team presents "Small-Scale Experiments: Are We There Yet?" (2026)

Important: everyone must read all the three papers to be prepared for the discussion. The presenting team will walk through the most important aspects of each paper, probably identifying parts that may be debatable, counter-intuitive, surprising, unclear, etc. The presenting team may use slides, but it is not strictly required.