Better Human Interaction Through Improved Reinforcement Learning

Travis Mandel: When new students come to play an educational game, how can we determine what content to give them such that they learn as much as possible?

February 21, 2017
4:30 pm - 5:30 pm
Location
Kemeny Hall 007
Sponsored by
Computer Science Department
Audience
Public
More information
Sandra Hall

When new students come to play an educational game, how can we determine what content to give them such that they learn as much as possible?  When a frustrated customer calls in to a helpline, how can we determine what to say to best assist him/her?  When an ill patient comes into the clinic, how do we determine what tests to run and treatments to give to maximize his/her quality of life?

These problems, though diverse, are all a seemingly natural choice for reinforcement learning, where an AI agent learns from experience how to make a sequence of decisions to maximize some reward signal.  However, unlike many recent successes of reinforcement learning, in these settings the agent gains experience solely by interacting with humans (e.g. game players or patients). As a result, although the potential to directly impact human lives is much greater, intervening to collect new data is often expensive and potentially risky .  Therefore, in this talk I present methods that allow us to evaluate candidate learning approaches offline using previously-collected data instead of actually deploying them. Further, I present new learning algorithms which ensure that, when we do choose to deploy, the data we gather is maximally useful. Finally, I explore how reinforcement learning agents should best leverage human expertise to gradually extend the capabilities of the system, a topic which lies in the exciting area of Human-in-the-Loop AI. 

Throughout the presentation I will discuss how I have deployed real-world experiments and used data from thousands of kids to demonstrate the effectiveness of our techniques in the domain of educational games.

Bio: Travis Mandel is a PhD Candidate at the University of Washington, co-advised by Zoran Popović and Emma Brunskill (Carnegie Mellon).  His research focuses on developing new reinforcement learning techniques that enable us to solve challenging real-world problems, particularly in the domain of educational games.   He obtained his master's degree in computer science and engineering from the University of Washington in 2013, and his bachelor's degree in computer science from Carnegie Mellon University in 2011.  He has received the AAAI-15 Outstanding Program Committee Award, CHI 2014 Honorable Mention, and EDM 2013 Best Paper Nomination.

Location
Kemeny Hall 007
Sponsored by
Computer Science Department
Audience
Public
More information
Sandra Hall