BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//Virginia Tech//VT Calendar//EN
BEGIN:VEVENT
DTSTAMP:20210218T193000Z
UID:1613487622566@events.msu.edu
CATEGORIES:Conferences / Seminars / Lectures
DTSTART:20210218T193000Z
DTEND:20210218T203000Z
SUMMARY:Math Seminar Series
DESCRIPTION:
 Applied Mathematics\n
 \n
 Mikhail Belkin , University 
 of California, San Diego, will be speaking.\n
 \n
 The 
 presentation is titled "A theory of 
 optimization and transition to linearity in 
 deep learning."\n
 \n
 Contact Olga Turanova (turanova@msu.edu) 
 for more information.\n
 \n
 The 
 success of deep learning is due, to a large extent, 
 to the remarkable effectiveness of gradient-based 
 optimization methods applied to large 
 neural networks. In this talk I will discuss 
 some general mathematical principles allowing 
 for efficient optimization in over-parameterized 
 non-linear systems, a setting that 
 includes deep neural networks. Remarkably, it 
 seems that optimization of such systems is "easy". 
 In particular, optimization problems corresponding 
 to these systems are not convex, 
 even locally, but instead satisfy locally the 
 Polyak-Lojasiewicz (PL) condition allowing 
 for efficient optimization by gradient descent 
 or SGD. We connect the PL condition of these 
 systems to the condition number associated 
 to the tangent kernel and develop a non-linear 
 theory parallel to classical analyses of over-parameterized 
 linear equations. In a related 
 but conceptually separate development, I will 
 discuss a new perspective on the remarkable 
 recently discovered phenomenon of transition 
 to linearity (constancy of NTK) in certain 
 classes of large neural networks. I will show 
 how this transition to linearity results from 
 the scaling of the Hessian with the size of 
 the network. Joint work with Chaoyue Liu and 
 Libin Zhu\n\n
 Price: free\n
 Sponsor: Department of Mathematics\n
 Sponsor's Homepage: https://www.math.msu.edu/\n
 Contact name: Department of Mathematics\n
 Contact phone: (517) 353-0844\n
 for more info visit the web at:\n 
 https://www.math.msu.edu/Seminars/CalendarView.aspx?month-of=September2016\n
LOCATION:Online via Zoom
END:VEVENT
END:VCALENDAR