Machine Learning Seminar by Emre Neftci (Forschungszentrum Jülich, UC Irvine)

Emre Neftci (Forschungszentrum Jülich, GER and Univ California Irvine, USA)

Meta-Training Neuromorphic Hardware with Surrogate Gradients

Continual learning at the edge is an aspirational goal of AI technologies.  Neuromorphic hardware is particularly attractive in this regard, thanks to its inherently local computing paradigm and its potential compatibility with future and emerging devices. With recent advances in training neuromorphic hardware using differentiable programming, it is now possible to achieve competitive accuracy and performance compared to Deep Neural Networks (DNNs). However, the data-intensive and iterative training procedure that powers DNNs is incompatible with the device non-idealities and real-time operation that characterize neuromorphic hardware. In this talk, I will argue that gradient-based meta-learning can play a critical role in closing this gap, enabling accurate and fast learning, even in the presence of severe device non-linearities, programming asymmetry, and endurance problems. Given that such non-idealities do not impact performance as much as previously thought, these our results have the potential to redefine the target metrics that guide the design of emerging devices.