Structured Dynamics for Visual Intelligence: Diffusion, Memory, and Causal Reasoning

Matthew Korban
UVA ECE

Time: 2026-10-28, 12:00 - 13:00 ET
Location: Rice 540 and Zoom

Abstract Visual learning systems must combine detailed observations, retain useful information over long sequences, and model how events unfold. These demands raise a common question: how should a model structure the flow of information? In this talk, I will present three complementary approaches from our recent work in IEEE Transactions on Pattern Analysis and Machine Intelligence and IEEE Transactions on Image Processing that connect visual learning with numerical analysis, Bayesian estimation, and causal modeling.

First, the Multigrid Anisotropic Diffusion Transformer (MGDT) replaces softmax attention with edge-preserving diffusion on token graphs. A multigrid hierarchy enables efficient global information exchange while preserving local distinctions. I will then turn to streaming video, where Kalmanized Recursive Least Squares Attention (KRLS-Attn) reformulates temporal aggregation as online regression. Recursive updates maintain a fixed-size memory and predictive uncertainty, supporting adaptive processing as observations accumulate. Finally, the Video-Causal State Space Model (VCSSM) incorporates a learned directed acyclic graph into latent dynamics, enabling structured prediction and the analysis of interventions and counterfactuals under explicit modeling assumptions.

Together, these methods illustrate a shared design principle: placing structure in how representations interact and evolve can improve computational efficiency, robustness, and interpretability. I will discuss the central intuitions, selected theoretical insights, and experimental evidence across image and video tasks, concluding with opportunities to bring efficient computation, adaptive memory, and causal reasoning together in future visual learning systems.

Bio Matthew Korban is a Postdoctoral Research Associate and Project Lead in the Department of Electrical and Computer Engineering at the University of Virginia. He received his Ph.D. in Electrical and Computer Engineering from Louisiana State University in 2021. His research focuses on computer vision, video understanding, multimodal learning, and efficient and trustworthy machine learning. His recent work develops diffusion-based transformers, streaming attention mechanisms, and causal state-space models. He leads research on NSF-supported video analysis for education and has published in IEEE TPAMI, IEEE TIP, Pattern Recognition, and ECCV. He is an IEEE Senior Member and serves as an Associate Editor for several Springer Nature journals.