Adversarial Robustness in Deep Learning NLP Systems

Abstract

The rise of deep learning Natural Language Processing (NLP) systems and their expanding real-world applications have heightened the importance of adversarial attacks and defenses for these systems. Research has demonstrated that adversaries employing text-based attacks can significantly reduce the performance of NLP models. In response, defensive strategies have been developed and shown to counteract some of the adverse effects. However, there is ample room for improvement on both fronts. Current attack methodologies often compromise the readability of text after an attack, indicating the need to refine these approaches to maintain coherence and fluency. On the other hand, defense mechanisms fail to adequately address the domain shift between standard and adversarial examples, leading to less effective protection. Moreover, they often require multiple steps of projected gradient descent during optimization, which prolongs the training duration. In this thesis, we aim to better understand the vulnerabilities of deep learning NLP models by investigating symbol-based attacks and defense mechanisms as well as word-level defense mechanisms.

Our first work introduces Special Symbol Text Attacks `SSTA’, a technique to improve the performance of language adversarial perturbations using special symbols that have downstream task information associated with them. We show that punctuation insertions, when limited to a few symbols (apostrophes and hyphens), are a superior attack vector compared to character insertions, and result in text that is easier and faster to read as assessed with the Test of Word Reading Efficiency (TOWRE). We also explore defense techniques such as adversarial training, utilization of grammar checkers, as well as basic defense strategies like the removal of all punctuation. Our findings indicate no single technique completely solves the problem, where each method possesses its own unique pros and cons.

With the emergence of better and more powerful word-level attacks, we next explore defense techniques aimed at word substitution attacks. We find that these attacks introduce a large domain shift in the data. Thus, we propose a novel adversarial training approach named Semantic robust defense (SemRoDe) a Macro Adversarial Training strategy to enhance the robustness of LMs. Drawing inspiration from recent studies in the image domain, we investigate and confirm that in a discrete data setting such as language, adversarial samples generated via word substitutions do indeed belong to an adversarial domain exhibiting a high Wasserstein distance from the base domain. Our method introduces a regularizer that aids the model learn a robust representation that bridges these two domains, increasing robustness to adversarial attacks.

Based on the above research, we explore how adversarial attacks translate to large language models. Currently, large language models with billions of parameters suffer from adversarial attacks just like their earlier, smaller counterparts. However, the threat models have changed. Previously, gray-box access, where input embeddings or output logits were visible to the user, might have been reasonable. With the introduction of closed-source models, no model information is available other than the generated output. This means that current black-box attacks can only utilize the final prediction to detect if an attack is successful. We investigate the possibility of using attack guidance, similar to using output probabilities, but while having only black-box access in a classification setting. We explore this by using the ability to elicit confidence from the model. We find that the elicited confidence is calibrated enough and not hallucinated for current LLMs, which allows us to extract a meaningful directional signal to perform black-box optimization. By minimizing the elicited confidence, we can then maximize the likelihood of a misclassification. Using this guidance as an attack vector can allow an attacker to optimize an input sample over multiple perturbations to find a local attack optimum, thus achieving stronger overall perturbations.

Bio of speaker
Brian Formento is a last-year PhD student in SOC working on adversarial attacks and defenses for deep learning NLP models. His research aims to better understand deep learning models by breaking them, with the goal of developing more effective adversarial robustness techniques in the future.