Physics x Data Science Seminar - Max Lovig - Yale University

The Dynamics Of CNN Networks---Applications vs (Simple) Theory

Many machine learning results and applications focus on understanding the model fit at the end of training, both in a quantitative sense (for example, the generalization error) and a qualitative one (for example, the features learned).

In the first part of this talk, I will show that the dynamics of model training can reveal deeper insights into the features the model learns in some simple applications. In the second part, I consider a simple but realistic CNN training setup and derive a set of order-parameter ODEs that track the training dynamics of the population loss. Simulations of these ODEs show that they accurately capture some of the phenomena observed in CNNs in practice.

Hosts: Naomi Gluck and Nathan Suri