Deep learning basics and the problem of implicit regularization (Benoit Dherin, Google)

08.12.2022 13:15

In the first part of this talk, we will recall the building blocks of deep learning, framing the learning problem as an optimization problem solved in practice by gradient descent. This first part will be very accessible and self-contained. Then we will attempt to convey how surprising it is that deep learning works so well given the extreme complexity of its solution space, pointing toward the existence of an implicit regularization mechanism self-selecting the simpler solutions that generalize best ahead of the more complex ones that do not perform well. At last, we will outline a recent approach attempting to uncover such an implicit regularization mechanism based on the backward error analysis of the gradient descent scheme.

Lieu

Bâtiment: Conseil Général 7-9

Room 1-15, Attn. unusual time, Séminaire "Topologie et géométrie"

Organisé par

Section de mathématiques

Intervenant-e-s

Benoit Dherin, Google

entrée libre

Classement

Catégorie: Séminaire