When
Where
Associate Professor of Operations & Decision Technologies, Judith Norman Davis and Kim G. Davis Professor of Business Analytics, Co-Director of Institute for Business Analytics, Kelly School of Business, Indiana University
Longitudinal Impact of Preference Biases on Recommender Systems' Performance
Abstract: Research studies have shown that recommender systems' predictions that are observed by users can cause biases in users' post-consumption preference ratings. This can happen as part of the standard, normal system use, where biases are typically caused by the system's inherent prediction errors (i.e., due to the less-than-perfect accuracy of recommendation methods). Because users' preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems' performance. Our simulation results show that preference biases significantly impair the system's prediction performance (i.e., prediction accuracy) as well as users' consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is non-linear to the size of the bias, i.e., large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Furthermore, given the impact of preference bias on the recommender systems' performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems.
Bio: Jingjing Zhang is an Associate Professor of Information Systems and holds the Judith Norman Davis and Kim G. Davis Professorship of Business Analytics at the Kelley School of Business, Indiana University. She earned her Ph.D. in Business Administration from the University of Minnesota in 2012. Her research interests include personalization techniques, recommender systems, human-computer interactions, mobile app marketplace, and emerging digital platforms. Jingjing's work has been published in leading academic journals such as MIS Quarterly, Information Systems Research, INFORMS Journal on Computing, IEEE Transactions on Knowledge and Data Engineering, ACM Transactions on Information Systems, and ACM Transactions on Computer-Human Interaction. Her research contribution has been recognized with numerous awards, including the 2023 Women in RecSys Journal Paper of the Year Award, the 2020 ISR Best Published Paper Award, the INFORMS ISS Sandra A. Slaughter Early Career Award, the Nunamaker-Chen Dissertation Award, and multiple Best Paper accolades at academic conferences. Currently, Jingjing serves as an Associate Editor for Information Systems Research. She has previously served as a Guest Associate Editor for esteemed journals such as MIS Quarterly, Decision Support Systems, and Journal of the Association for Information Systems. Her editorial contribution was acknowledged with the AE of the Year Award from ISR in 2022 and the Best AE Award at ICIS 2019.