Advances in Conversational and Time-Sensitive Question Answering
Abstract:
This thesis focuses on two key aspects of question answering (QA): conversational question answering and time-sensitive question answering. In conversational question answering, we investigate dialogue breakdown detection, a critical task for maintaining coherent and effective conversations. For time-sensitive question answering, we tackle the challenge of addressing questions related to evolving knowledge updates in a dynamic and fast-changing environment. Together, these efforts contribute to enhancing the adaptability and reliability of question answering systems in diverse real-world scenarios.
The first work in this thesis focuses on advancing smooth communication in conversational question answering, where we develop novel models that enhance a system's ability to recognize breakdowns in dialogues. We use multi-lingual transfer learning with co-attention networks for the task of dialogue breakdown detection. We propose a novel neural model that jointly incorporates a pre-trained cross-lingual language model and a co-attention network to tackle this task. To overcome the problem of limited task-specific training data, we utilize training data from multiple language sources with pre-trained cross-lingual language models. We also apply a co-attention network to capture the interaction between the previous dialogue history and the last utterance. Our proposed method significantly outperforms prior work on the Dialogue Breakdown Detection Challenge 4 (DBDC4) dataset.
Next, we apply semi-supervised learning to tackle the problem of limited labeled data in dialogue breakdown detection. We propose a novel semi-supervised teacher-student learning framework with two teachers which are trained on labeled data and perturbed labeled data respectively. We enrich the training data with unlabeled data which are jointly assigned pseudo-labels by the two teachers, to train a student model. Through our proposed training approach, the student achieves performance improvements over single teacher. Experimental results on the Dialogue Breakdown Detection Challenge dataset DBDC5 and Learning to Identify Follow-Up Questions dataset LIF show that our approach outperforms all prior published approaches as well as other supervised and semi-supervised baseline methods.
Lastly, given the increasing demand for systems capable of handling rapidly evolving knowledge, we study the problem of time-sensitive question answering. Large language models (LLMs), typically pre-trained on static datasets with fixed knowledge cut-off dates, often fail to produce accurate answers when real-world facts change. To address this challenge, we construct a dynamic QA dataset, DynaQuest, that reflects knowledge updates over time based on changes in Wikipedia Infoboxes. The dataset is automatically updated through a scalable pipeline and serves as a benchmark for evaluating the ability of LLMs to integrate recent information. Experimental results show that existing models struggle with this task, especially in the absence of retrieved context. To improve performance, we propose CARL, a context-aware reinforcement learning method that guides the model to better incorporate retrieved knowledge while balancing it with parametric knowledge. CARL encourages answer generation that is concise, evidence-grounded, and aligned with up-to-date information, leading to consistent improvements over strong baselines in retrieval-augmented generation settings.
By addressing these challenges, our research aims to maintain dialogue coherence and incorporate updated knowledge, leading to better question answering performance.