Events Mini Calendar

Go to previous month Go to next month
  S M T W R F S
W40 27 28 29 30 1 2 3
W41 4 5 6 7 8 9 10
W42 11 12 13 14 15 16 17
W43 18 19 20 21 22 23 24
W44 25 26 27 28 29 30 31

Today is:
Wed, Jul 24, 2024

MSU Events Calendar

 Choose Calendar view:   Day   Week   Month     Search   Filter

Math Seminar Series  (Conferences / Seminars / Lectures)

Applied Mathematics

Rene Vidal from Johns Hopkins University will be speaking.

The presentation is titled "On the Regularization Properties of Structured Dropout."

Dropout and its extensions (e.g. DropBlock and DropConnect) are popular heuristics for training neural networks, which have been shown to improve generalization performance in practice. However, a theoretical understanding of their optimization and regularization properties remains elusive. This talk will present a theoretical analysis of several dropout-like regularization strategies, all of which can be understood as stochastic gradient descent methods for minimizing a certain regularized loss. In the case of single hidden-layer linear networks, we will show that Dropout and DropBlock induce nuclear norm and spectral k-support norm regularization, respectively, which promote solutions that are low-rank and balanced (i.e. have factors with equal norm). We will also show that the global minimizer for Dropout and DropBlock can be computed in closed form, and that DropConnect is equivalent to Dropout. We will then show that some of these results can be extended to a general class of Dropout-strategies, and, with some assumptions, to deep non-linear networks when Dropout is applied to the last layer.

more information...

Location: Zoom link provided on Math Seminars page [map]
Price: free
Sponsor: Department of Mathematics
Contact: Department of Mathematics
phone pic (517) 353-0844