Maxime Krier, TU Kaiserslautern

Title:
Bridging Neural Network Architectures and Numerical Differential Equations

Abstract:

Over the past few years, neural networks have become one of the most important tools for supervised machine learning. A class of architectures called ResNets have shown good results on tasks like image classification. We show that ResNet architectures can be interpreted as a numerical discretization of ordinary differential equations. A common problem that arises in training neural networks is the poor generalization to new data. We approach this problem by applying the well examined concept of stability of ODEs to ResNets. In this talk we discuss the construction of a network architecture based on a numerical scheme for solving ODEs. Then we analyze how stability properties of the numerical scheme can be used to deduce characterizations of robustness of the neural network. Based on our results we propose improvements to ResNet architectures that ensure better generalization ability of the network with respect to small perturbations in the data.