This is a great point and shows how world models are limited by a lack of memory, and by the fact that memory is in an inefficient and expensive format (video frames containing pixels). What would the ideal kind of memory be? It would be essentially a scene graph (James Clark, 1976!) hierarchically describing all of the objects in a world together with their properties, in a way that’s easy to query and operate on. This is the reason for the core thesis that the ideal evolutionary path sees a merging of world models and engines, the AI side providing vast amounts of loosely organized audiovisual and textual knowledge, and the engine providing consistent reproducible data representation and simulation.