Novel Approaches and Evaluation for Robust Machine Reading Comprehension
Dr Ng Hwee Tou, Provost'S Chair Professor, School of Computing
COM1 Level 2
SR5, COM1-02-01
Abstract:
The field of machine reading comprehension (MRC) has witnessed significant advancements, enabling the extraction of valuable insights from textual content and the ability to answer complex questions. These developments hold immense promise across various applications such as information retrieval and intelligent assistants. However, the robustness of MRC models, which refers to their ability to accurately and consistently respond to diverse and challenging inputs, remains a pressing concern.
While traditional evaluation metrics offer valuable insights, they often fall short in capturing model performance under challenging conditions. To tackle this issue, our first work introduces a novel sub-question evaluation metric specifically designed for multi-hop MRC tasks. This metric sheds light on the underlying reasoning paths where top-performing MRC models struggle.
In our second work, we delve into enhancing the robustness of MRC models against linguistic variations, adversarial attacks, and out-of-distribution test data. We propose a contrastive regularized approach that effectively strengthens MRC models' capabilities to handle unseen challenging test sets.
Furthermore, we shift our focus towards improving the robustness of spoken MRC systems across diverse English accents. To achieve the goal, we construct a novel large-scale multi-accent SRC dataset, incorporating naturally recorded spoken questions. We show challenges arising from modality gaps and accent variations by observing degradation in the performance with cascaded models. We propose a novel end-to-end SRC model to bridge the modality gap and facilitates semantic interaction between speech and text features, proving more robust to diverse accents.
In conclusion, through evaluating multi-hop reasoning, proposing a contrastive regularized approach for unseen challenging questions, and exploring spoken MRC across diverse accents, we take a step forward towards more robust and reliable MRC models.