Obvious, but: billions of dollars in compute that OpenAI has been expending on training largely went to automated experiments of this sort, with larger scale models. I really wonder how many Xe^23-24 artifacts they've burned through.