Foundation Models for Dynamical Systems
Research Overview
Foundation models pretrained on large amounts of data have shown impressive performance across NLP and computer vision. This research line investigates whether the same paradigm applies to dynamical systems — can models pretrained on vast numbers of randomly generated systems generalize to unseen target systems?
We explore transformer-based architectures pretrained entirely on synthetic data from randomly generated ordinary differential equations. Our findings show surprising zero-shot generalization to new target systems of various complexities for both state prediction and control tasks. We also investigate combining these foundation models with classical estimation techniques like the Unscented Kalman Filter (UKF) for state estimation.
This work is funded by the JUPITER AI Factory (JAIF), part of the EuroHPC initiative.