BEGIN:VCALENDAR
VERSION:2.0
CALSCALE:GREGORIAN
PRODID:iCalendar-Ruby
BEGIN:VEVENT
CATEGORIES:Lectures & Talks
DESCRIPTION:Join the Goergen Institute for Data Science for Principled Fram
eworks for Designing Deep Learning Models: Efficiency\, Robustness\, and Ex
pressivity with Tan Nguyen\, postdoctoral scholar with the Department of Ma
thematics at UCLA. Lunch will be provided to attendees.\n\nAbstract: Design
ing deep learning models for practical applications\, including those in co
mputer vision\, natural language processing\, and mathematical modeling\, i
s an art that often involves an expensive search over candidate architectur
es. In this talk\, I present novel frameworks to facilitate the process of
designing efficient and robust deep learning models with better expressivit
y via three principled approaches: optimization\, differential equation\, a
nd statistical modeling. \n\nFrom an optimization viewpoint\, I leverage th
e continuous limit of the classical momentum accelerated gradient descent t
o improve Neural ODEs training and inference. The resulting Momentum Neural
ODEs accelerate both forward and backward ODE solvers\, as well as allevia
ting the vanishing gradient problem (Efficiency). \nFrom a differential equ
ation approach\, I present a random walk interpretation of graph neural net
works (GNNs)\, revealing a potentially inevitable over-smoothing phenomenon
. Based on this random walk viewpoint of GNNs\, I then propose the graph ne
ural diffusion with a source term (GRAND++) that overcomes the over-smoothi
ng issue and achieves better accuracy in low-labeling rate regimes (Robustn
ess). \n\nUsing statistical modeling as a tool\, I show that the attention
in transformer models can be derived from solving a nonparametric kernel re
gression problem. I then propose the FourierFormer\, a new class of transfo
rmers in which the softmax kernels are replaced by the novel generalized Fo
urier integral kernels. The generalized Fourier integral kernels can automa
tically capture the dependency of the features of data and remove the need
to tune the covariance matrix (Expressivity). \n\nBio: Dr. Tan Nguyen is c
urrently a postdoctoral scholar in the Department of Mathematics at the Uni
versity of California\, Los Angeles\, working with Dr. Stanley J. Osher. He
obtained his Ph.D. in Machine Learning from Rice University\, where he was
advised by Dr. Richard G. Baraniuk. Dr. Nguyen is an organizer of the 1st
Workshop on Integration of Deep Neural Models and Differential Equations at
ICLR 2020. He also had two awesome long internships with Amazon AI and NVI
DIA Research. He is the recipient of the prestigious Computing Innovation P
ostdoctoral Fellowship (CIFellows) from the Computing Research Association
(CRA)\, the NSF Graduate Research Fellowship\, and the IGERT Neuroengineeri
ng Traineeship. He received his M.S. and B.S. in Electrical and Computer En
gineering from Rice University in May 2018 and May 2014\, respectively.\n\n
This seminar is part of the tenure-track\, Assistant Professor in Data Scie
nce faculty search led by the Goergen Institute for Data Science.
DTEND:20230221T180000Z
DTSTAMP:20240527T041318Z
DTSTART:20230221T170000Z
GEO:43.128881;-77.629774
LOCATION:Wilson Commons\, Gowen Auditorium 213
SEQUENCE:0
SUMMARY:Principled Frameworks for Designing Deep Learning Models: Efficienc
y\, Robustness\, and Expressivity
UID:tag:localist.com\,2008:EventInstance_42455375727242
URL:https://events.rochester.edu/event/principled_frameworks_for_designing_
deep_learning_models_efficiency_robustness_and_expressivity
END:VEVENT
END:VCALENDAR