[2018-07-09] Dr. Le Song, Georgia Institute of Technology, “Reinforcement Learning as Saddle Points”

Poster:Post date:2018-06-12
Title: Reinforcement Learning as Saddle Points  
 Date: 2018-07-09 11:00-12:00
Location: R210, CSIE
Speaker: Dr. Le Song, Georgia Institute of Technology
Hosted by: Prof. Chih-Jen Lin




We revisit the Bellman equation in reinforcement learning, and reformulate it into a novel saddle point optimization problem using Nesterov’s smoothing technique and the Legendre-Fenchel transformation. We then develop a new algorithm, called Smoothed Bellman Error Embedding, to solve this optimization problem where any differentiable function class may be used for the policy. We provide what we believe to be the first convergence guarantee for general nonlinear function approximation, and analyze the algorithm’s sample complexity. Empirically, our algorithm compares favorably to state-of-the-art baselines in several benchmark control problems. https://arxiv.org/pdf/1712.10285.pdf
Smoothed Dual Embedding Control - arxiv.org                      
Smoothed Dual Embedding Control Bo Dai 1, Albert Shaw , Lihong   
Li3, Lin Xiao2, Niao He4, Jianshu Chen2, Le Song1 1 Georgia      
Insititute of Technology, 2 Microsoft Research, Redmond     



Le Song is an Associate Professor in the College of Computing, and an Associate Director of the Center for Machine Learning, Georgia Institute of Technology. He is now on sabbatical leave, and working at Ant Financial as a Principal Engineer. Le received his Ph.D. in Machine Learning from University of Sydney and NICTA in 2008, and then conducted his post-doctoral research in the Department of Machine Learning, Carnegie Mellon University, between 2008 and 2011. His principal research direction is machine learning, especially nonlinear models, such as kernel methods and deep learning, and probabilistic graphical models for large scale and complex problems, arising from artificial intelligence, network analysis and other interdisciplinary domains. His work on distribution embedding has received US National Science Foundation CAREER Award’14. Furthermore, he is also the recipient of a number of best paper awards, including the NIPS'17 Workshop on ML for Molecule and Materials Best Paper Award, Recsys’16 Deep Learning Workshop
Best Paper Award, AISTATS'16 Best Student Paper Award, IPDPS'15
Best Paper Award, NIPS’13 Outstanding Paper Award, and ICML’10
Best Paper Award. He has also served as the area chair or senior program committee for many leading machine learning and AI conferences such as ICML, NIPS, AISTATS, AAAI and IJCAI. He is also the action editor for JMLR, and associate editor for IEEE PAMI.


Last modification time:2018-06-12 PM 4:59

cron web_use_log