Towards Trustworthy Natural Language Processing
Dr Qizhe Xie, Assistant Professor, School of Computing
COM1 Level 3
MR1, COM1-03-19
Abstract:
This thesis examines the privacy, robustness, and alignment of language models through a series of studies. In the first, we compare the current state-of-the-art definitions of privacy in machine learning to human ideas of privacy for natural language. We conclude that language requires better definitions of privacy, both to better protect privacy and to offer interpretable guarantees to users.
In our second study, we examine the robustness of current LLMs, uncovering a vulnerability in current LLMs that causes models to respond to harmful requests after a single character perturbation. We find that this is due to how these characters are tokenized and the contexts in which these characters appear in the training data. This points toward a need to carefully assess all components of models for robustness, and improve the robustness of current LLMs.
In our third study, we propose an efficient new defense against adversarial attacks on LLMs, using self-evaluation. We find that LLMs without any fine-tuning are able to reliably detect adversarial attacks and offer strong defenses against them, dramatically reducing the attack success rates of various attacks.
In our final work, we compare the robustness of an alignment method that uses in-context learning instructions for pre-trained LLMs rather than fine-tuning. We observe taht that this method is able to not only give aligned model outputs, but is resistant to jailbreak attacks in many settings at levels that are comparable to models that have undergone RLHF tuning. Additionally, the prompt-based nature of the method makes it well-suited for adapting to new attacks and for improving alignment for low-resource languages where data for fine-tuning is hard to come by.
We conclude by emphasizing the importance of flexibility, ease of implementation, efficiency, and collaboration with other fields in improving trustworthy NLP. Though many technical advances have been made in improving privacy, robustness, and fairness of models, improving trust is a holistic process that must consider practitioners, users, and policy-makers.