Compositional Object-Centric Representations for Robust Visual Perception
Dr Lee Mong Li Janice, Professor, School of Computing
COM1 Level 3
MR1, COM1-03-19
Abstract:
Despite significant advancements, real-world deployment of modern vision models in critical applications remains hindered by limitations including poor out-of-distribution generalization, failure in occluded settings, reliance on large numbers of high-quality data, and lack of interpretability. We posit that these limitations stem from fundamental differences in structural processing between modern vision models and the human visual cortex. Although modern vision models emulate the hierarchical structure of the human visual system, they continue to process images as holistic units. In contrast, human vision relies on the unsupervised learning of objects as compositional building blocks, which allow for robust recognition, generalization, occlusion-aware perception, and sample-efficient learning.
To bridge this gap, this thesis proposes a unified object-centric representation framework comprising three stages: discovery, representation, and application of object representations. The proposed models demonstrate enhanced robustness, sample efficiency, occlusion-aware perception, and out-of-distribution generalization. OC-Net overcomes limitations of pixel-based reconstruction in unsupervised object-centric learning using a feature connectivity algorithm and novel object-centric regularization terms optimized for downstream generalization. ECO-Net introduces explicit graph representations of object parts, a novel co-part object discovery algorithm, and a memory module, addressing failures of prior methods on multi-part objects in occluded or out-of-distribution settings. Finally, DP-GAT integrates these insights by leveraging OC-Net for the unsupervised discovery of fine-grained biological objects and combining it with ECO-Net's graph-based relationship modeling, achieving superior performance in image-based disease progression prediction and 3D tumor segmentation. Together, these contributions pave the way for the reliable deployment of vision models in critical real-world applications.