Foundation Models for Dynamical Systems
Research Overview
Foundation models pretrained on large amounts of data have shown impressive performance across NLP and computer vision. This research line investigates whether the same paradigm applies to dynamical systems — can models pretrained on vast numbers of randomly generated systems generalize to unseen target systems?
We study transformers pretrained on synthetic data generated from dynamics functions sampled from a reproducing kernel Hilbert space. The initial work evaluates prediction and adaptation to unseen systems, including cart-pole and Furuta pendulum experiments in simulation and hardware. Fine-tuning improves performance on new systems. We also combine these models with an Unscented Kalman Filter for state estimation, evaluated on a containership benchmark with varying dynamics.
This work is funded by the JUPITER AI Factory (JAIF), part of the EuroHPC initiative.