BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//Virginia Tech//VT Calendar//EN
BEGIN:VEVENT
DTSTAMP:20200924T183000Z
UID:1600867426202@events.msu.edu
CATEGORIES:Conferences / Seminars / Lectures
DTSTART:20200924T183000Z
DTEND:20200924T193000Z
SUMMARY:Math Seminar Series
DESCRIPTION:
 Rebecca Willett from the University of Chicago 
 will be speaking.\n
 \n
 The presentation is titled 
 "Regularization in Infinite-Width ReLU Networks."\n
 \n
 This 
 meeting will take place online.\n
 \n
 The 
 seminar description is as follows:\n
 \n
 A 
 growing body of research illustrates that 
 neural network generalization performance 
 is less dependent on the network size (i.e. number 
 of weights or parameters) and more dependent 
 on the magnitude of the weights. That is, 
 generalization is not achieved by limiting 
 the size of the network, but rather by explicitly 
 or implicitly controlling the magnitude 
 of the weights. To better understand this phenomenon, 
 we will explore how neural networks 
 represent functions as the number of weights 
 in the network approaches infinity. Specifically, 
 we characterize the norm required to realize 
 a function f as a single hidden-layer ReLU 
 network with an unbounded number of units 
 (infinite width), but where the Euclidean norm 
 of the weights is bounded, including precisely 
 characterizing which functions can be realized 
 with finite norm. This was settled for 
 univariate functions in Savarese et al. (2019), 
 where it was shown that the required norm 
 is determined by the L1-norm of the second derivative 
 of the function. We extend the characterization 
 to multivariate functions (i.e., 
 networks with d input units), relating the required 
 norm to the L1-norm of the Radon transform 
 of a (d+1)/2-power Laplacian of the function. 
 This characterization allows us to show 
 that all functions in certain Sobolev spaces 
 can be represented with bounded norm and to 
 obtain a depth separation result. These results 
 have important implications for understanding 
 generalization performance and the distinction 
 between neural networks and more traditional 
 kernel learning.\n\n
 Price: free\n
 Sponsor: Department of Mathematics\n
 Sponsor's Homepage: https://www.math.msu.edu/\n
 Contact name: Department of Mathematics\n
 Contact phone: (517) 353-0844\n
 for more info visit the web at:\n 
 https://www.math.msu.edu/Seminars/CalendarView.aspx?month-of=September2016\n
LOCATION:Zoom Link  found in link on Math Seminars Page
END:VEVENT
END:VCALENDAR