When
Where
Ruijiang Gao
PhD Candidate, McCombs School of Business, University of Texas at Austin
Learning Robust Complementary Policies for Human-AI Teams
Abstract: Human-AI complementarity is important when neither the algorithm nor the human yields dominant performance across all instances in a given context. Recent work that explored human-AI collaboration has considered decisions that correspond to classification tasks. However, in many important contexts, humans undertake courses of action. In this paper, we propose a framework for a novel human-AI collaboration for selecting advantageous courses of action, which we refer to as Learning Complementary Policies for Human-AI teams. Our solution aims to exploit the human-AI complementarity to maximize decision rewards by learning both an algorithmic policy that aims to complement humans by a routing model that defers decisions to either a human or the AI to leverage the resulting complementarity. We then extend our approach to leverage opportunities and mitigate risks that arise in practice: 1) when a team is composed of multiple humans with differential and potentially complementary abilities, 2) when the observational data includes consistent deterministic actions, 3) when the covariate distribution of future decisions differ from that in the historical data, and 4) when there are unobserved confounders that only humans have access to. We demonstrate the effectiveness of our proposed methods using data on synthetic and real human responses, and find that our methods offer reliable and advantageous performance across settings, and that it is superior to when either the humans or the AI make decisions on their own.
Bio: Ruijiang Gao is a PhD Candidate at McCombs School of Business, UT Austin. His research focuses on Human-Centered Machine Learning. Ruijiang has contributed to algorithmic development in robust personalized decision rules with human-AI teams, AI ethics, uncertainty quantification, and causal machine learning. His work has been published in top-tier machine learning venues (ICML/AISTATS/AAAI/IJCAI), and invited for revisions at top business journals such as Management Science and Operations Research, and has received a Best Student Paper award at the Conference on Information Systems and Technology. Ruijiang was a visiting researcher at Harvard Business School and has interned at Netflix, IBM, Amazon, and Tencent.