Machine Learning Seminar Bojian Yin, Institute of Automation, Chinese Academy of Sciences in Beijing

Beyond Backpropagation: Scaling Local Learning to LLMs

When
1 dec 2026 from 2 p.m. to 1 dec 2026 3 p.m. CET (GMT+0100)
Where
CWI, L202
Add

Beyond Backpropagation: Scaling Local Learning to LLMs

Abstract: Backpropagation has powered modern deep learning, but its reliance on global gradient propagation creates challenges for memory efficiency and parallelization. Can we train large language models without it? In this talk, I will introduce SOLO (Shared-Output Local Learning), a simple approach that enables neural networks to learn through local objectives rather than global backpropagation. We show that local learning can scale to billion-parameter language models, achieving competitive performance while reducing memory requirements and improving training throughput. Our findings challenge the conventional wisdom that global backpropagation is essential for training large language models, opening new directions for more efficient and scalable AI.