This part of the Karpathy/Dwarkesh interview was great. "Humans collapse during the course of their lives...we end up saying more and more of the same stuff, and the learning rates go down..." Many older people I admire have designed their lives to avoid this happening to them

It seems that when you're younger, weight updates happen kind of naturally. As you age you have to do this explicitly, which means: 1) reading and writing a lot, to make updates "clearer" 2) having conversations with people, esp people who disagree with you

And on the reading, at least some of the time you need to be sampling from places you wouldn't normally look. Otherwise the natural tendency is just to read more of whatever reinforces your existing views.

This also happens if you scroll too much / are "too online" btw: your thoughts collapse to the same set of uninteresting thoughts as everyone else. Important to mix in other, uncorrelated influences such as old books and lots of time talking with smart "offline" people.