Contextual Control: Integrating Learning, Optimization, and Physical Structures in Control Systems

Learning dynamical models from high-dimensional observations (e.g., images) is central to deploying model predictive control (MPC) when the system state is not directly observed. Recent latent predictive approaches, including joint-embedding predictive architectures (JEPAs), learn compact state representations for prediction and planning without reconstruction. However, these representations are often sensitive to nuisance variations—such as background changes or visual distractors—leading to degraded closed-loop performance under distribution shift. We address this by augmenting the predictive objective with a bisimulation-based constraint that enforces control-relevant state abstraction: observations with similar transition dynamics and outcomes are mapped to nearby latent states. This yields representations better aligned with MPC, filtering out control-irrelevant variability while preserving predictive structure. Across all benchmarks, our model consistently improves robustness to slow features while operating in a reduced latent space, up to 10× smaller than that of DINO-WM.

James Anderson is an Associate Professor of Electrical Engineering at Columbia University and is also a member of the Data Science Institute. From 2016 to 2019, he was a senior postdoctoral scholar in the Department of Computing + Mathematical Sciences at the California Institute of Technology. Prior to Caltech, he held a Junior Research Fellowship at St John’s College, University of Oxford, and was affiliated with the Department of Engineering Science. He received his DPhil (PhD) from Oxford in 2012 and his BSc and MSc degrees from the University of Reading in 2005 and 2006, respectively. His research spans control, learning theory, and optimization, with applications in smart grids and energy markets. Together with his students and collaborators, he has received several best paper awards in venues such as the IEEE Transactions on Control of Network Systems, the IEEE Conference on Decision and Control, and the Learning for Dynamics and Control Conference.

Graphical characterizations of uni-variable systems, such as Nyquist and Bode diagrams, are taught in almost all control curricula and widely used across engineering domains. These tools are intuitive and informative, enabling many closed-loop properties, including stability and robustness, to be inferred by inspection. However, for multivariable systems, the coupling of multi-dimensional information poses challenges as well as opens up many possibilities for extending such graphical tools. In this talk, we will examine several emerging tools developed in this direction, including the scaled relative graph and the Davis–Wielandt shell, along with their associated feedback stability certificates. In particular, we will highlight the connections among these tools and their relationship to quadratic separations. We will also outline the key ideas behind their visualization and the validation of their resulting robust stability conditions. Finally, we will present a case study on congestion control protocols and discuss how scalability could be incorporated into these tools for analyzing large-scale networked systems.

Ding Zhang is an incoming Research Fellow at The Australian National University. He received the B.Eng. degree in Mechanical Engineering from Huazhong University of Science and Technology, Wuhan, China, and the Ph.D. degree in Electronic and Computer Engineering from The Hong Kong University of Science and Technology, Hong Kong SAR, where he also worked as a Postdoctoral Research Fellow. He was a visiting student with the EMAN Group at King Abdullah University of Science and Technology, Thuwal, Saudi Arabia, and with the Control Group at University of Cambridge, Cambridge, United Kingdom. His research focuses on graphical stability analysis of large-scale networked systems, with broader interests in matrix theory, spectral graph theory, and control theory.