Letian (Zac) Chen

EDUCATION

Master of Science: Computer Science
Georgia Institute of Technology
GPA: 4.0/4.0

Bachelor of Science: Psychology
Peking University
GPA: 3.78/4.0

Bachelor of Science: Computer Science and Technology
Peking University
GPA: 3.80/4.0

RESEARCH

Learning from Suboptimal Demonstration via Self-Supervised Reward Regression
Graduate Research Assistant, Advisor: Matthew Gombolay, Georgia Tech

• Characterized policy performance degradation from noise injection by a sigmoid function.
• Proposed a novel IRL framework, SSRR, to learn policies that are better than suboptimal demonstrations by inferring the idealized reward function (i.e., the latent intent of the demonstrator).
• Proposed Noisy-AIRL to enhance the robustness of SSRR by providing more reliable initial reward via training with noise.
• Applied algorithm on three virtual robotic tasks and a real robot table-tennis task; achieved accurate recovery of the demonstrator intention and a better-than-best-demonstration policy.

Robot Learning from Heterogeneous Demonstration
Master Thesis, Advisor: Matthew Gombolay, Georgia Tech

• Dissected and categorized the heterogeneous demonstration problems.
• Designed multiple algorithms for learning from heterogeneous strategy and heterogeneous performance demonstrations.
• Showed the algorithms could achieve better learning from heterogeneous strategy demonstration than assuming homogeneity or dividing data to achieve homogeneity.
• Illustrated the algorithms could achieve better performance than suboptimal demonstrations and previous learning-from-suboptimal-demonstration techniques.

Joint Inference of Task Reward and Strategy Reward
Graduate Research Assistant, Advisor: Matthew Gombolay, Georgia Tech

• Modeled humans' latent objective via shared task reward and individual strategy reward.
• Proposed a novel IRL framework, MSRD, to jointly infer task reward and strategy reward to gain a better estimation of both.
• Applied algorithm on two virtual robot control tasks and one real robot table-tennis task; achieved better learning of task reward than SOTA AIRL, extracted precise strategic rewards, and optimized versatile policies that resemble the heterogeneous demonstrations.

Model-Free and Model-Based Algorithms in Human Sequential Decision Making
Undergraduate Thesis, Advisor: Hang Zhang, Peking University

• Designed an experiment to investigate human’s learning strategy (model-free vs model-based) under multi-task setting.
• Showed hybrid model with forgetting mechanism best explain subject data via computational model comparison; Confirmed conclusion by simulation that learned hybrid model recovered subject behavior.
• Still exploring meta-learning computational model explanation.

Better Exploration using Good and Bad Demos
Directed Research, Advisor: Yizhou Wang, Peking University

• Introduced new algorithm built on Bayesian Neural Network and Thompson Sampling.
• Proposed sample efficiency proof for the new method based on Gaussian Process and Thompson Sampling.
• Developed tool to record human demonstrations on OpenAI Universe platform.

AWARDS

Graduate of merit in Beijing

Excellent Graduate in Peking University

Zhang Wenjin Scholarship link

Scholarship for Undergraduate Research

First Prize of National Olympiad in Informatics in Provinces Advanced Group

WORK

Reinforcement Learning Intern
iRobot Corporation

• Identified real-world challenges of Offline Policy Evaluation (OPE) methods.
• Created a ease-to-use benchmark dataset where real-world challenges present.
• Proposed an ad-hoc OPE algorithm selection method via validation mechanisms.

Teaching Assistant
Georgia Institute of Technology

• Assisted CS 7648 Interactive Robot Learning

Teaching Assistant
Georgia Institute of Technology

• Assisted OMSCS 7641 Machine Learning

Research Assistant
Georgia Institute of Technology

• Assisted Professor Matthew Gombolay
• Worked on multi-agent expected policy gradient and sampling-based policy gradient from May 2019 - Aug 2019
• Worked on heterogeneous inverse reinforcement learning and robot table tennis from Sep 2019 - Dec 2019
• Worked on learning from suboptimal demonstration from Dec 2019 - Aug 2019

Teaching Assistant
Georgia Institute of Technology

• Assisted OMSCS 7641 Machine Learning

iOS Engineer
Peking University PKU Helper Team

• Developed and maintained iOS app “PKU Helper” for Peking University campus life (10k+ users)
• Information and download link: PKU Helper

Teaching Assistant
Peking University

• Assisted Professor Jun Sun in Introduction to Computation
• Designed practice sets, held office hour, set exam papers