Guozhang Chen from Peking University is visiting CWI. His focus is on brain-inspired computing, with expertise in such diverse topics like learning in recurrent neural networks, spiking neural network models of cortical processing, and mechanisms of visual attention. His work on SpikeStereoNet uses SNNs for stereo dpeth estimation.
Title: Global credit assignment via dynamical criticality
Abstract: Efficiently training recurrent neural networks on long sequences remains an open challenge. The standard global paradigm, backpropagation through time (BPTT), suffers from vanishing and exploding gradients and memory costs that scale linearly with sequence length. Conversely, biologically inspired local learning rules are memory-efficient but typically introduce severe bias. To bridge this gap, we introduce Criticality-driven Online Local Alignment (COLA). By leveraging the long-range spatiotemporal correlations inherent to the critical regime, COLA enables a strictly local learning rule to approximate global error propagation, thereby combining online efficiency with gradient descent precision. Theoretically, for a recurrent neural network with $H$ hidden units, COLA requires only an $\bigO{H}$ auxiliary state and constant activation memory, completely independent of sequence length. Empirically, COLA is competitive with BPTT on standard benchmarks and demonstrate!
s superior robustness on stability-sensitive tasks. Finally, we conduct a rigorous analysis of the approximation error to provide a theoretical foundation for reliable online learning.
