From Task-Specific Architectures to Task-Agnostic Generalization: A Path Towards Spatio-Temporal Foundation Models
COM1 Level 3
MR1, COM1-03-19
Abstract:
Spatio-temporal data represents information that evolves across both space and time, such as traffic flows, air quality measurements. It is critical for understanding dynamic real-world systems. However, existing approaches to analyzing spatio-temporal data rely heavily on task-specific neural networks, which, despite strong performance, struggle to scale and generalize across different tasks and domains.
This thesis addresses the challenge of building more generalizable and scalable models by bridging task-specific modeling with the emerging paradigm of foundation models. It first develops advanced task-specific architectures DualSTN and STGNP to uncover key design principles, namely attention mechanisms for stronger representation learning and probabilistic modeling for better uncertainty estimation and generalization. Building on these insights, the thesis proposes unified frameworks: diffusion-based models (USTD), prompt-driven Transformer architectures (STGP), and multi-modal masked autoencoders (ST-MAE), that jointly learn across tasks and domains. Together, these contributions provide a pathway toward scalable, general-purpose spatio-temporal foundation models capable of capturing complex spatial and temporal dependencies.