Stefano Peluchetti (Cogent Labs Tokyo)

“Deep neural networks and stochastic processes”

We review deep neural networks and their training, and then focus on the connection between neural networks and stochastic processes. Deep neural networks at initialization correspond to prior models in function space, and under appropriate assumptions they converge to Gaussian processes in the limit of infinite width. While this connection holds “a priori”, it is possible to derive similar results for “a posteriori” training via gradient descent. In this talk we discuss these and related results, and we introduce a recent line of research which generalizes established results to non-Gaussian initializations and stochastic processes arising in the limit of infinite depth.