On instabilities, paradoxes and universal barriers in AI

Date

Sep 24, 2020

Time

11:00 AM - 12:00 PM

Speaker

Anders Hansen

Affiliation

University of Cambridge, UK

Series

MPI-CBG Thursday Seminar

Language

en

Main Topic

Biologie

Host

Florian Jug

Description

Deep learning has had unprecedented success in the sciences, however, suffers from a non-negligible Achilles heel: instability. The instability phenomenon in deep learning is universal across different fields ranging from computer vision and image classification, to voice and audio recognition, via automated diagnosis in medicine, to inverse problems and imaging. We will discuss this highly complex issue and provide mathematical explanations for the phenomenon. Intriguingly, the reasons for the instabilities vary depending on the application. However, a common phenomenon is that the instabilities are not caused by the lack of approximation power of neural networks. Indeed, it is a paradox that despite the unstable trained networks, there will typically exist other stable and accurate networks for the same applications. The problem is that the training process does not construct them. Herein lies a fascinating barrier. Despite the rich collection of results from approximation theory regarding existence of neural network with powerful approximation and stability guaranties, it can be shown that many of these networks cannot be computed by a computer regardless of computing power. Thus, theoretical results a la “there exists a neural network with the following properties” do not mean that such a network can ever be computed on a digital computer. We are therefore left with the fundamental question: can stable and accurate neural networks be computed for the many problems where deep learning is currently used, or is instability a necessary artefact in AI?

Last modified: Sep 25, 2020, 12:09:26 AM