LLMs really need a 'copy and paste' function with a scratchpad, insane amounts of manual copying going on where it copies stuff 'through' the model (at inference token speed). its driving me crazyyyyyy.

New episode w @AdamMarblestone on what the brain's secret sauce is: how do we learn so much from so little? Also, the answer to Ilya’s question: how does the genome encode desires for high level concepts that are only seen during lifetime? Turns out, they’re deeply connected questions. Timestamps 0:00:00 – The brain’s secret sauce is the reward functions, not the architecture 0:22:20 – What the genome actually encodes 0:42:42 – What kind of RL is the brain doing? 0:50:31 – Is biological hardware a limitation or an advantage? 1:03:59 – Why we need to map the human brain 1:23:28 – What value will automating math have? 1:38:18 – Architecture of the brain Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!