any workable theory of superintelligence must respect physical and information-theoretic bounds. there is much literature in control theory and dynamical systems that deals with this that seems totally unknown to communities concerned with asi.

it's incoherent to talk about planning, simulation, reasoning, recursive self-improvement etc. in the abstract without referring to concrete physical, informational, and computational bounds. this would be a very interesting research program to boot!

even if you insist that an asi will find 'cheat codes in reality', we can at least say something about where those cheat codes might have to be found, know something about the implications of finding them, and be able to look for them in advance.

this isn't intended as a callout, i think there's something really interesting / important here.

harry potter and the methods of integrable systems