The trajectory of the future could soon get set in stone. In a new paper, I look at mechanisms through which the longterm future's course could get determined within our lifetimes. These include the creation of AGI-enforced institutions, a global concentration of power, the widespread settlement of space, the first immortal beings, the widespread design of new beings, and the ability to self-modify in significant and lasting ways. I’m not very confident that such events will occur, but in my view they’re likely enough to make work to steer them in better directions very valuable. Let’s take each mechanism in turn. First, AGI-based institutions. Once we have AGI, decision-makers could: - Set up an institution, and align the AGI so that it understands that constitution and has the enforcement of that constitution as its goal. - Empower that AGI with the ability to enforce the constitution. - Store copies of the neural weights of the AGI in multiple locations in order to reduce the risk of destruction of any one of the copies. - Reload the original Constitutional-AGI to check that any AGIs that are tasked with ensuring compliance with the constitution maintain adherence to their original goals as those AGIs learn and update their neural weights over time. This would be as if, rather than having the Supreme Court interpret the US Constitution, we could conjure up the ghosts of Madison and Hamilton and ask them directly - and their views were decisive. With these in place, this AGI-enforced constitution could operate indefinitely. Second, immortality. Throughout history, death has functioned as a natural brake on the persistence of any particular set of values or power structures. Over time, even the most entrenched values eventually change as new generations replace the old. Post-AGI technology could fundamentally alter this dynamic. Digital beings would inherently be immune to biological aging; when combined with perfect replication and hardware migration, we’ll be able to create the minds whose exact values and decision-making processes could persist unchanged indefinitely. A similar dynamic could hold for biological immortality. A technological explosion driven by AGI could dramatically extend or effectively eliminate biological constraints on human lifespans through technologies targeting the fundamental mechanisms of aging. Third, designing beings. Through history, change has happened in part because successive generations do not inherit the same values as their forebears. But this dynamic could change after AGI. Probably, the vast majority of beings that we create will be AI, and they will be products of design — we will be able to choose what preferences they have. And, with sufficient technological capability, we would likely be able to choose the preferences of our biological offspring, too. Even if people choose not to live forever, their values could continue to persist through perfect transmission from one generation to the next. Fourth, strong self-modification. In the future, people will probably be able to modify their own beliefs and preferences such that they can precisely choose what beliefs and preferences to have. So, not only might people today be able to control society’s future values by living forever; they would also be able to control the values of their future selves. A religious zealot might choose to have unshakeable certainty that their favoured religion is true; an ideological extremist might choose to have an irrevocable and unwavering preference in favour of their political party over any other. As well as creating new mechanisms that enable persistent path-dependence, a post-AGI world could also reduce the causes of disruption, too. Throughout history, societal changes have often been driven by technological innovations that disrupt existing power structures. However, as civilisation approaches technological maturity—the hypothetical point at which all major technologies have been invented—this source of disruption would disappear. Advanced technology would help prevent other sorts of disruption, too. It would dramatically improve prediction capabilities: advanced AI systems could process vastly more information, model complex systems with greater precision, and forecast outcomes over longer time horizons. So it would be much less likely people would relinquish their influence just by making some mistake. Finally, a post-AGI world might be characterised by indefinite defense-dominance, enabling a permanently stable concentration of power. In particular, indefinite defense-dominance could come about as a result of widespread space settlement. If star systems are strongly defense-dominant, then the starting distribution of star systems could, in principle, be held onto indefinitely. It might be that, after the initial allocation, there is trade or gifting of some star systems; but even if so, there would still be very strong path-dependence, as the final allocation of star systems would be extremely influenced by the starting allocation. These issues might seem like far-off concerns - but the intelligence and industrial explosions make them near-term. I think it’s over 1 in 3 that we see an intelligence explosion starting in the next 10 years. And if advanced AI results in explosive technological progress and industrial expansion, then many of the new mechanisms for persistence will arrive in quick succession.

What about for *my* use-cases? I have a couple of private very hard benchmark Qs (one philosophy, one finance) and.... GPT-5-Pro did clearly the best of any model. I was really impressed. Between this and Gemini-2.5-Pro, I feel that genuine "philosophy-assistant" AI is almost within reach.

Reddit *hates* the GPT-5 release. (This is just one highly-upvoted thread among many.)

We live in the midst of many moral catastrophes, all at once. Technological and material abundance has given us the opportunity to create a truly flourishing world, and we have utterly squandered that opportunity. Will the same be true of the future?

1/ 🚨 New report out! Futures with Digital Minds: Expert Forecasts in 2025 Together with Bradford Saad, I surveyed experts on the future of digital minds — computers capable of subjective experience. Here’s why this is important and what they said 👇

Super interesting paper. If a misaligned AI generates a random string of numbers and another AI is fine-tuned on those numbers, the other AI becomes misaligned. But only if both AIs start from the same base model. This has consequences for preventing secret loyalties: - If an employee fine-tunes GPT-5 to be secretly loyal to them, they could then generate innocuous-seeming data and fine-tune all other GPT-5 copies to be secretly loyal (e.g. by inserting the data in further post-training) - BUT this technique wouldn't work to make GPT-6 secretly loyal in the same way (I doubt this technique would actually work for smg as complex as a sophisticated secret loyalty, but that's the implication of the pattern here if i've understood correctly)

I harangued @AricFloyd for years trying to get him to do a youtube channel... but 2M+ views on the first go? Dang, that's impressive. (It's also a great video - an overview of @DKokotajlo's AI2027 - check it out!)

Excited to announce my new research organisation, Fivethought! At Fivethought, we're focused on how to navigate the transition from “transitioning to a world with superintelligent AI systems” to “a world with superintelligent AI systems”. (1/n)

*New* @givewell podcast - We share some rough estimates on forecasted aid cuts: - $60b in global health funding pre-cuts - $6b of that is extremely cost-effective (twice our current "10x" bar, ~$3k/death averted) - 25% cuts forecasted in these extremely cost-effective programs