Structured Dynamics for Visual Intelligence: Diffusion, Memory, and Causal Reasoning

Matthew Korban
UVA ECE

Time: 2026-10-28, 12:00 - 13:00 ET
Location: Rice 540 and Zoom

Abstract Visual learning systems must combine detailed observations, retain useful information over long sequences, and model how events unfold. These demands raise a common question: how should a model structure the flow of information? In this talk, I will present three complementary approaches from our recent work in IEEE Transactions on Pattern Analysis and Machine Intelligence and IEEE Transactions on Image Processing that connect visual learning with numerical analysis, Bayesian estimation, and causal modeling.

First, the Multigrid Anisotropic Diffusion Transformer (MGDT) replaces softmax attention with edge-preserving diffusion on token graphs. A multigrid hierarchy enables efficient global information exchange while preserving local distinctions. I will then turn to streaming video, where Kalmanized Recursive Least Squares Attention (KRLS-Attn) reformulates temporal aggregation as online regression. Recursive updates maintain a fixed-size memory and predictive uncertainty, supporting adaptive processing as observations accumulate. Finally, the Video-Causal State Space Model (VCSSM) incorporates a learned directed acyclic graph into latent dynamics, enabling structured prediction and the analysis of interventions and counterfactuals under explicit modeling assumptions.

Together, these methods illustrate a shared design principle: placing structure in how representations interact and evolve can improve computational efficiency, robustness, and interpretability. I will discuss the central intuitions, selected theoretical insights, and experimental evidence across image and video tasks, concluding with opportunities to bring efficient computation, adaptive memory, and causal reasoning together in future visual learning systems.

Bio Matthew Korban is a Postdoctoral Research Associate in the Department of Electrical and Computer Engineering at the University of Virginia, specializing in computer vision with a focus on video understanding. He has published in top-tier journals and conferences, including IEEE Transactions on Pattern Analysis and Machine Intelligence, Pattern Recognition, and ECCV. Matthew is an IEEE senior member and serves as an Associate Editor for several respected journals, including Signal, Image and Video Processing, The Visual Computer, and Multimedia Systems. In addition to his research contributions, he has held leadership roles, notably organizing the highly successful UVA Postdoc Symposium from 2021 to 2024.