Learning to Attend to Classify

Supervisor: Sander Bohté

In humans, attention focuses neural resources on a limited part of the sensory experience. Psychophysics also tells us that we only learn about that to which we attend. In deep learning, attention models are typically applied to sequence learning, where attention dynamically masks part of the stream [1]. Can we model the biological kind of attention to learn more efficiently?

References: [1] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998-6008).