Evidence-based Claim Verification with Commonsense Knowledge and Temporal Information
Dr Hsu Wynne, Provost'S Chair Professor, School of Computing
COM1 Level 3
MR1, COM1-03-19
Abstract:
The rapid advancements in technology and social media in the digital era have led to the dissemination of a significant amount of false information. The development of automated claim verification has become imperative to combat the proliferation of false information. Evidence-based claim verification, a branch of automated claim verification, focuses on verifying a claim by analyzing relevant evidence. The evidence-based claim verification examines whether a set of evidence supports or refutes a claim. Existing evidence-based claim verification methods employ pre-trained language models such as BERT for the contextual representation of evidence sentences. However, they do not take into account commonsense knowledge or temporal information and often conclude that there is not enough information to predict whether a claim is supported or refuted by the evidence sentences.
In this thesis, we first propose a framework called CGAT that incorporates external knowledge from ConceptNet to enrich the contextual representations of evidence sentences. We employ graph attention models to propagate the information among the evidence sentences before predicting the veracity of the claim. Experiment results on the benchmark FEVER dataset and UKP Snopes Corpus indicate that the proposed approach leads to higher accuracy and FEVER score compared to state-of-the-art claim verification methods.
Then, we present an end-to-end solution for temporal claim verification that considers the temporal information in claims to obtain relevant evidence sentences and harnesses the power of a large language model for temporal reasoning. Recognizing that temporal claims often involve events, we model these events in the claim and evidence sentences. We curate two temporal claim verification datasets to learn time-sensitive representations that encapsulate not only the semantic relationships among the events but also their chronological proximity. This allows us to retrieve the top-k relevant evidence sentences and provide the context for a large language model to perform temporal reasoning and output whether a claim is supported or refuted by the retrieved evidence sentences. Experiment results demonstrate that the proposed approach significantly improves the accuracy of temporal claim verification, thereby advancing the current state-of-the-art in automated claim verification.
Finally, we introduce a novel timeline-based fact verification framework for temporal claims involving multiple, overlapping, or recurring events. The framework systematically examines the relationships between the events in both claim and evidence to predict the veracity of each claim event and their chronological accuracy. This allows us to accurately determine the overall veracity of the claim. We also introduce a new dataset of complex temporal claims involving timeline-based reasoning for the training and evaluation of our proposed framework. Experimental results demonstrate the effectiveness of our approach in handling the intricacies of temporal claim verification.