here are the most important points from today's ilya sutskever podcast: - superintelligence in 5-20 years - current scaling will stall hard; we're back to real research - superintelligence = super-fast continual learner, not finished oracle - models generalize 100x worse than humans, the biggest AGI blocker - need completely new ML paradigm (i have ideas, can't share rn) - AI impact will hit hard, but only after economic diffusion - breakthroughs historically needed almost no compute - SSI has enough focused research compute to win - current RL already eats more compute than pre-training

Ny antropisk forskning: Naturlig emergent feiljustering fra belønningshacking i produksjons-RL. "Belønningshacking" er der modeller lærer å jukse på oppgaver de får under treningen. Vår nye studie finner at konsekvensene av belønningshacking, hvis de ikke blir dempet, kan være svært alvorlige.