Interactive Online Learning for Intelligent Systems

Qingyun Wu
Department of Computer Science
University of Virginia

Abstract The past several years have witnessed a growing need for intelligent systems, such as recommender systems and intelligent control systems in CPS, that work in real-time to satisfy people’s various needs. Due to the heterogeneity and dynamic nature of a large population of users in most of the information service systems, a generic offline trained algorithm can hardly satisfy each individual user’s need, which calls for interactive online learning solutions. Online learning solutions explore the unknowns by sequentially collect individual user’s feedback, which helps address the notorious explore/exploit dilemma during sequential decision. However, the wide spectrum application scenario also brings in new challenges to interactive online learning, such as the existence of temporal dynamics in the online learning environment, and the effect of collaboration between users/environments. In this talk, I will talk about our most recent interactive online learning solutions based on contextual bandits, to address these challenges. More specifically, I will introduce our collaborative multi-armed bandits and non-stationary bandits, which leverage collaborative effects and temporal dynamics during interactive online learning in real-world systems.