Toward a Theory of AI Evaluation

Han Shao
University of Maryland, College Park

Time: 2026-10-14, 12:00 - 13:00 ET
Location: Rice 540 and Zoom

Abstract

AI evaluations inform which models we develop, deploy, and trust. But what conclusions do these evaluations actually support? Answering this question requires understanding both what we aim to measure and what information the available data provide. In this talk, I will discuss recent work on the theoretical foundations of evaluating generative AI.

Bio:

Han Shao is an assistant professor of computer science at the University of Maryland, College Park. Her research spans machine learning, economics and computation, and algorithmic game theory. She is particularly interested in the theoretical foundations of AI evaluation and in how social and adversarial behavior shapes learning. Previously, she was a postdoctoral fellow at Harvard University’s Center of Mathematical Sciences and Applications, hosted by Cynthia Dwork and Ariel Procaccia. She received her Ph.D. in computer science from the Toyota Technological Institute at Chicago, advised by Avrim Blum.