BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//Virginia Tech//VT Calendar//EN
BEGIN:VEVENT
DTSTAMP:20201001T183000Z
UID:1601477475238@events.msu.edu
CATEGORIES:Conferences / Seminars / Lectures
DTSTART:20201001T183000Z
DTEND:20201001T193000Z
SUMMARY:Math Seminar Series
DESCRIPTION:
 Applied Mathematics\n
 \n
 Rene Vidal from Johns Hopkins 
 University will be speaking.\n
 \n
 The presentation 
 is titled "On the Regularization Properties 
 of Structured Dropout."\n
 \n
 Dropout 
 and its extensions (e.g. DropBlock and DropConnect) 
 are popular heuristics for training neural 
 networks, which have been shown to improve 
 generalization performance in practice. However, 
 a theoretical understanding of their optimization 
 and regularization properties remains 
 elusive. This talk will present a theoretical 
 analysis of several dropout-like regularization 
 strategies, all of which can be understood 
 as stochastic gradient descent methods 
 for minimizing a certain regularized loss. In 
 the case of single hidden-layer linear networks, 
 we will show that Dropout and DropBlock 
 induce nuclear norm and spectral k-support norm 
 regularization, respectively, which promote 
 solutions that are low-rank and balanced (i.e. 
 have factors with equal norm). We will also 
 show that the global minimizer for Dropout 
 and DropBlock can be computed in closed form, 
 and that DropConnect is equivalent to Dropout. 
 We will then show that some of these results 
 can be extended to a general class of Dropout-strategies, 
 and, with some assumptions, 
 to deep non-linear networks when Dropout is 
 applied to the last layer.\n\n
 Price: free\n
 Sponsor: Department of Mathematics\n
 Sponsor's Homepage: https://www.math.msu.edu/\n
 Contact name: Department of Mathematics\n
 Contact phone: (517) 353-0844\n
 for more info visit the web at:\n 
 https://www.math.msu.edu/Seminars/CalendarView.aspx?month-of=September2016\n
LOCATION:Zoom link provided on Math Seminars page
END:VEVENT
END:VCALENDAR