Robust Representation Learning from Imperfect Data

Abstract:

Modern artificial intelligence models are powered by massive data. Over the past decade, people have witnessed the growth of data by orders of magnitude, from small academic datasets to a collection of almost all the sources on the web. Although extensive effort has been continually paid to increase the data scale, the data itself, as well as the complementary labels, can never be perfect. On the one hand, the quality of individual samples in a dataset usually varies greatly, leading to difficulty in pattern recognition with a single solution. On the other hand, the annotations for the data, either from human or machines, inevitably carry errors or are even missing. Thus, how to learn robust representations in the presence of ubiquitous imperfect data poses a big challenge to researchers and becomes a key to developing powerful and safe AI models.

In this dissertation, we present research on robust representation learning in three important fields, which are self-supervised learning, domain adaptation, and AI medical diagnosis. We begin by introducing a novel design for learning robust representations from unlabeled data. We propose ContrastiveCrop which is tailored to craft better positive views for contrastive learning. As a plug-and-play module, ContrastiveCrop consistently improves the quality and robustness of learned representations when applied to multiple contrastive learning frameworks. Next, we present our work on learning robust representations from out-of-domain data. We propose a new framework termed BETA which tackles the problem of domain adaptation of black-box predictors. Through error suppression and domain division, the proposed framework greatly improves the robustness of the model on a wide range of domains. Finally, we explore how to apply the technique of robust representation learning in real-world medical scenarios. We aim to develop AI models that could communicate fine-grained knowledge from medical images like human experts, rather than simple binary outcomes.