On Foundation Models for Dynamical Systems from Purely Synthetic Data
preprint
A transformer pretrained on synthetic RKHS-sampled dynamics for prediction, generalization and data-efficient adaptation to new systems.
Resources
Summary
The paper studies whether foundation models for dynamical systems can be trained without large collections of real trajectories. A transformer is pretrained on synthetic data generated from dynamics functions sampled from a reproducing kernel Hilbert space. Evaluation focuses on prediction and adaptation to unseen systems, including cart-pole and Furuta pendulum experiments in simulation and hardware. Fine-tuning improves performance on new systems, with results examining generalization, data efficiency and robustness.
Citation
@article{ziegler2024foundation,
author = {Ziegler, Martin and Posada-Moreno, Andres Felipe and Solowjow, Friedrich and Trimpe, Sebastian},
date = {2024-11-30},
eprint = {2412.00395},
eprinttype = {arxiv},
title = {On Foundation Models for Dynamical Systems from Purely Synthetic Data},
url = {https://arxiv.org/abs/2412.00395}
}