We're launching an "AI psychiatry" team as part of interpretability efforts at Anthropic!  We'll be researching phenomena like model personas, motivations, and situational awareness, and how they lead to spooky/unhinged behaviors. We're hiring - join us!

the openai IMO news hit me pretty heavy this weekend i'm still in the acute phase of the impact, i think i consider myself a professional mathematician (a characterization some actual professional mathematicians might take issue with, but my party my rules) and i don't think i can answer a single imo question ok, yes, imo is its own little athletic subsection of math for which i have not trained, etc. etc., but. if i meet someone in the wild who has an IMO gold, i immediately update to "this person is much better at math than i am" now a bunch of robots can do it. as someone who has a lot of their identity and their actual life built around "is good at math," it's a gut punch. it's a kind of dying. like, one day you discover you can talk to dogs. it's fun and interesting so you do it more, learning the intricacies of their language and their deepest customs. you learn other people are surprised by what you can do. you have never quite fit in, but you learn people appreciate your ability and want you around to help them. the dogs appreciate you too, the only biped who really gets it. you assemble for yourself a kind of belonging. then one day you wake up and the universal dog translator is for sale at walmart for $4.99 the IMO result isn't news, exactly. in fact, if you look at the METR agent task length over time plot, i think agents being able to solve ~ 1.5 hour problems is coming right on time. so in some way we should not be surprised. and indeed, it appears multiple companies have achieved the same result. it's just... the rising tide rising as fast as it has been rising of course, grief for my personal identity as a mathematician (and/or productive member of society) is the smallest part of this story multiply that grief out by *every* mathematician, by every coder, maybe every knowledge worker, every artist... over the next few years... it's a slightly bigger story and of course, beyond that, there is the fear of actual death, which perhaps i'll go into more later. this package -- grief for relevance, grief for life, grief for what i have known -- isn't unique to the ai age or anything like that. i think it is a standard thing as one appreaches end of career or end of life. it just might be that that is coming a bit sooner for many of us, all at once. i wonder if we are ready

IT MAKES SENSE NOW. This xAI employee is openly OK with AI causing human extinction. Reminder: As horrifying as this is, ~10% of AI researchers believe this. It is NOT a fringe view! Unbelievably, even Turing Award winner Richard Sutton has repeatedly argued that extinction would be the MORALLY RIGHT thing to happen, if AIs were smarter than us (!) He goes around the world giving speeches saying we must "prepare for succession" and it "behooves us to bow out" and not stand in the way of "evolutionary progress" (even if it causes human extinction). Instead of crowds gasping in horror, he gets applause. Threaten one person? "You need help." Threaten 100? "Call the police!" Threaten millions? "Monster!" But build machines to end ALL of humanity and lecture ppl on about how it's a good thing? "Oooh, what a fascinating philosophical position! Have a Turing award!" So of course they advocate against safety - they're literally anti-human! They will NEVER stop or proceed cautiously unless they're restrained by civil society.

58% of people agree that "if we build AI models smarter than us, we will inevitably lose control over them". Just 12% disagree.