Towards Effective Time-Travelling Visualization System for Machine Learning Models Surveillance
COM3 Level 2
MR21, COM3 02-61
Abstract:
Deep Neural Networks (DNNs) have become integral to various domains, including safety-critical domains like healthcare and autonomous driving because of their impressive performance. However, their susceptibility to errors, especially in contexts where safety is paramount, poses significant risks. This underscores the urgent need to improve these models' interpretability and debugging capabilities, ensuring the identification and rectification of errors.
Monitoring the execution trace is a key technique for bug localization in traditional software. By viewing DNNs as analogous to computer programs, monitoring their execution trace is a natural approach for interpreting and debugging these often opaque models. Yet, this approach presents challenges with DNNs, as their abstract, high-dimensional features like representations and decision boundaries are more difficult to track than the structured variables found in conventional software. Employing dimension reduction techniques could provide a direct method to visualize these intricate data layers. This thesis is dedicated to creating a visual debugging system that equips users with advanced tools for error diagnosis and correction, surpassing the capabilities of existing methods.
First, we term the visualization of a sequence of extremely high-dimensional representations as "time-traveling" visualization. The challenges lie in maintaining the representation structure and preserving the temporal continuity of representation evolution. To this end, we propose a time-traveling visual solution DVI, aiming to manifest accurate spatio-temporal relationships of training dynamics in the low-dimensional space. Technically, we propose four spatial and temporal properties and design our visualization solution to satisfy them. These properties preserve the most important information when (inverse-)projecting input samples between the visible low-dimensional and the invisible high-dimensional space, for causal analyses. We show that DVI can well reflect the characteristics of various training scenarios, showing the good potential of DVI as a debugging tool for analyzing deep learning training processes.
Second, in response to certain inefficiencies and visualization errors associated with DVI, we introduce an enhanced version of time-traveling visualization named TimeVis. While DVI was groundbreaking in visualizing training dynamics, its limitations in efficiency and accuracy impeded its broader practical application. To address this, we have formulated a technique termed "temporality spatialization". This approach seamlessly integrates both spatial relationships (e.g., adjacent samples within a single epoch) and temporal relationships (e.g., the same sample across consecutive training epochs) within a unified high-dimensional topological framework. This spatio-temporal complex empowers us to swiftly and accurately train a singular visualization model, facilitating precise projections and inverse projections of data across epochs, irrespective of their dimensionality. Comprehensive experiments indicate that, when benchmarked against DVI, TimeVis excels not only in preserving the visualized time-travelling semantics but also boasts a remarkable 15-fold increase in visualization speed.
Third, monitoring the training of neural networks is essential for identifying potential data anomalies, enabling timely interventions and conserving significant computational resources. To this end, we propose SentryCam to monitor the training of neural networks. SentryCam is an automated, live-update visualization tool that reveals the progression of hidden representations during training. Our results show that this visualization offers a more comprehensive view of the learning dynamics compared to basic metrics such as loss and accuracy over various datasets. Furthermore, we show that SentryCam can provide deeper insights into tasks such as task transfer and catastrophic forgetting in continual learning settings.
Finally, despite the training dynamics of the high-level concepts such as sample representations and classification boundaries are now observable, the model developers can still be overwhelmed by tens of thousands of moving dots across hundreds of training epochs (i.e., frames in the animation), which makes them miss important training events. We propose DeepDebugger to recommend the samples of user interest in a human-in-the-loop manner. On the one hand, DeepDebugger monitors the training dynamics of samples and recommends suspicious samples based on their abnormality. On the other hand, our recommendation is interactive and fault-resilient for the model developers to explore the training process. By learning users' feedback, DeepDebugger refines its recommendation to fit their intention. Our user study of the tool shows that the interactive recommendation of DeepDebugger can help the participants accomplish the debugging tasks by saving completion time and boosting performance.