Bayesian Optimisation Techniques for High-Dimensional and Adversarial Settings
Abstract:
Bayesian Optimisation (BO) is an increasingly essential approach for sequentially optimizing black-box functions without a closed-form solution or those that are costly to evaluate. BO is applied to solve optimization problems in real-world applications, such as hyperparameter tuning of machine learning models, robotics, automated model selection, and more. The training of AlphaGo, where BO tuned the neural network's hyperparameters leading to significant improvements in playing strength, is a notable example of its application. However, the efficacy of BO is impacted by problem dimensionality and robustness issues. Addressing these two challenges is crucial and the subject of active research, which can enable the broader application of BO.
We discuss the application of Bayesian Optimization (BO) in high-dimensional and adversarial settings. The first part focuses on scaling BO to high dimensions by studying various approaches in the literature and proposing a method that extends existing work. The second part focuses on the settings where BO algorithms are subject to adversarial attacks and analyzes conditions for which an adversary can succeed, proposing several attacks and defenses. Finally, we apply BO to perform practical adversarial attacks on Convolutional Neural Networks (CNN), with considerations for the high-dimensional nature of finding adversarial perturbations. The presentation will conclude by discussing the latest research built on this work and future research directions.