Dead wrong. Model sizes are going to get much bigger in 2026 as a result of larger scale-up worlds in Nvidia racks (and Google TPU ones too), going from B200s with 1.4 TB per 8-chip scale-up world to GB300 with 20 TB per scale-up world.
expatanon
expatanonMar 18, 03:04
Altman admitted that transformer models have hit the wall. Most improvements in the last 9 months are attributable more to the tooling around the model rather than the models themselves. In other words, this technology is rapidly maturing with no signs of another leap.
The jawdropping datacenter investments of 2024/2025 are starting to come online now, with much better hardware than current-gen models are optimized for inference on.
94