I just had the pleasure of reading a quite brilliant essay by @tainab. It makes an incredibly simple point (though wraps it in a bit of social theory that one can take or leave at preference): there is one very basic way in which the data foundation models are trained in are

biased: they oversample loquacity in the same way that looking at companies in the S and P 500 today creates survivorship bias in retrospective technical analysis or (ironically) there is an anthropic bias when thinking about possible worlds. Sampling based on tokens v

based on people’s responses over samples the loquacious.

Most people most of the time generate very few tokens.