BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//Virginia Tech//VT Calendar//EN
BEGIN:VEVENT
DTSTAMP:20201001T183000Z
UID:1601477475238@events.msu.edu
CATEGORIES:Conferences / Seminars / Lectures
DTSTART:20201001T183000Z
DTEND:20201001T193000Z
SUMMARY:Math Seminar Series
DESCRIPTION:
Applied Mathematics\n
\n
Rene Vidal from Johns Hopkins
University will be speaking.\n
\n
The presentation
is titled "On the Regularization Properties
of Structured Dropout."\n
\n
Dropout
and its extensions (e.g. DropBlock and DropConnect)
are popular heuristics for training neural
networks, which have been shown to improve
generalization performance in practice. However,
a theoretical understanding of their optimization
and regularization properties remains
elusive. This talk will present a theoretical
analysis of several dropout-like regularization
strategies, all of which can be understood
as stochastic gradient descent methods
for minimizing a certain regularized loss. In
the case of single hidden-layer linear networks,
we will show that Dropout and DropBlock
induce nuclear norm and spectral k-support norm
regularization, respectively, which promote
solutions that are low-rank and balanced (i.e.
have factors with equal norm). We will also
show that the global minimizer for Dropout
and DropBlock can be computed in closed form,
and that DropConnect is equivalent to Dropout.
We will then show that some of these results
can be extended to a general class of Dropout-strategies,
and, with some assumptions,
to deep non-linear networks when Dropout is
applied to the last layer.\n\n
Price: free\n
Sponsor: Department of Mathematics\n
Sponsor's Homepage: https://www.math.msu.edu/\n
Contact name: Department of Mathematics\n
Contact phone: (517) 353-0844\n
for more info visit the web at:\n
https://www.math.msu.edu/Seminars/CalendarView.aspx?month-of=September2016\n
LOCATION:Zoom link provided on Math Seminars page
END:VEVENT
END:VCALENDAR