BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//Virginia Tech//VT Calendar//EN
BEGIN:VEVENT
DTSTAMP:20210218T193000Z
UID:1613487622566@events.msu.edu
CATEGORIES:Conferences / Seminars / Lectures
DTSTART:20210218T193000Z
DTEND:20210218T203000Z
SUMMARY:Math Seminar Series
DESCRIPTION:
Applied Mathematics\n
\n
Mikhail Belkin , University
of California, San Diego, will be speaking.\n
\n
The
presentation is titled "A theory of
optimization and transition to linearity in
deep learning."\n
\n
Contact Olga Turanova (turanova@msu.edu)
for more information.\n
\n
The
success of deep learning is due, to a large extent,
to the remarkable effectiveness of gradient-based
optimization methods applied to large
neural networks. In this talk I will discuss
some general mathematical principles allowing
for efficient optimization in over-parameterized
non-linear systems, a setting that
includes deep neural networks. Remarkably, it
seems that optimization of such systems is "easy".
In particular, optimization problems corresponding
to these systems are not convex,
even locally, but instead satisfy locally the
Polyak-Lojasiewicz (PL) condition allowing
for efficient optimization by gradient descent
or SGD. We connect the PL condition of these
systems to the condition number associated
to the tangent kernel and develop a non-linear
theory parallel to classical analyses of over-parameterized
linear equations. In a related
but conceptually separate development, I will
discuss a new perspective on the remarkable
recently discovered phenomenon of transition
to linearity (constancy of NTK) in certain
classes of large neural networks. I will show
how this transition to linearity results from
the scaling of the Hessian with the size of
the network. Joint work with Chaoyue Liu and
Libin Zhu\n\n
Price: free\n
Sponsor: Department of Mathematics\n
Sponsor's Homepage: https://www.math.msu.edu/\n
Contact name: Department of Mathematics\n
Contact phone: (517) 353-0844\n
for more info visit the web at:\n
https://www.math.msu.edu/Seminars/CalendarView.aspx?month-of=September2016\n
LOCATION:Online via Zoom
END:VEVENT
END:VCALENDAR