1/ The standard x + f(x) residual connection is the bedrock of modern architectures. It is also a massive bottleneck. Unweighted accumulation causes state magnitudes to grow linearly, diluting early layers and capping efficient depth scaling. 🧵