In the last 6 months the NanoGPT Speedrun to 3.28 loss on FineWeb dropped by 33% to 2mins. Recently a subset of these changes were bulk copy-pasted to the larger-scale 2.92 loss track. Surprisingly, the untuned yolo run broke the 2.92 loss record by 25%.