CWI Lecture Professor Marta Kwiatkowska (University of Oxford)

We cordially invite you to attend the CWI lecture given by Professor Marta Kwiatkowska (Fellow of Trinity College Department of Computer Science, University of Oxford) entitled 'Safety Verification of Deep Neural Networks'.
  • What Formal Methods English
  • When Jun 09, 2017 from 11:00 AM to 12:00 PM (Europe/Amsterdam / UTC200)
  • Where Room L120 at CWI, Science Park 123 in Amsterdam
  • Contact Name
  • Contact Phone 020-592 4189
  • Add event to calendar iCal

Everyone is welcome to attend the CWI lecture given by Marta Kwiatkowska.

Prof.dr  Jos Baeten Prof.dr. Jan Rutten

Safety Verification of Deep Neural Networks
Marta Kwiatkowska
University of Oxford

Deep neural networks have achieved impressive experimental results in image classification, but can surprisingly be unstable with respect to adversarial perturbations, that is, minimal changes to the input image that cause the net-work to misclassify it. With potential applications including perception modules and end-to-end controllers for self-driving cars, this raises concerns about their safety. We develop a novel automated verification framework for feed-forward multi-layer neural networks based on Satisfiability Modulo Theory (SMT). We focus on image manipulations, such as scratches or changes to camera angle or lighting conditions, and define safety for an image classification decision in terms of invariance of the classification with respect to manipulations of the original image within a region of images that are close to it. We enable exhaustive search of the region by employing discretisation, and propagate the analysis layer by layer. Our method works directly with the network code and, in contrast to existing methods, can guarantee that adversarial examples, if they exist, are found for the given region and family of manipulations. If found, adversarial examples can be shown to human testers and/or used to fine-tune the network. We implement the techniques using Z3 and evaluate them on state-of-the-art networks, including regularised and deep learning networks. We also compare against existing techniques to search for adversarial examples and estimate network robustness.

Marta Kwiatkowska is Professor of Computing Systems and Fellow of Trinity College, University of Oxford. She led the development of the PRISM model checker (, the leading software tool in the area and winner of the HVC Award 2016. Applications of probabilistic model checking have spanned communication and security protocols, nanotechnology designs, power management, game theory, planning and systems biology, with genuine flaws found and corrected in real-world protocols. Kwiatkowska gave the Milner Lecture in 2012 in recognition of "excellent and original theoretical work which has a perceived significance for practical computing" and was awarded an honorary doctorate from KTH Royal Institute of Technology in Stockholm in 2014. Her research has been supported by the ERC Advanced Grant VERIWARE "From software verification to everyware verification" and the EPSRC Programme Grant on Mobile Autonomy. She is a Fellow of ACM, Member of Academia Europea and Fellow of EATCS.