For folks running autoresearch: here are the top 10 findings from 20+ agents across 1000+ experiments. 1. Step count dominated everything 2. A simple attention pattern consistently won 3. Initialization turned out to matter more than optimizer tweaks 4. The swarm discovered a “make it learnable” principle 5. The architecture sweet spot was surprisingly small 6. Many improvements were actually just noise 7. Some common techniques failed badly 8. Research roles emerged organically 9. The biggest opportunity might still be unexplored 10. Collective memory accelerated discovery 1️⃣ Step count dominated everything The single most important discovery: More optimizer steps consistently beat larger batches. Halving batch size from 2^19 → 2^18: • doubled training steps • improved BPB by 0.007 Later the swarm revisited batch 2^17. Earlier experiments showed it was too noisy, but once the architecture improved, it became optimal and helped push the final result to 0.9631. This suggests something subtle: Optimal batch size depends on model quality. Better architectures tolerate more gradient noise....