Design and Analysis of Network-on-Chip based Hardware Accelerator Architectures for Neural Network Applications
COM3 Level 2
MR21, COM3 02-61
Abstract:
Deep learning applications have been revolutionizing multiple tasks in different fields such as robotics, computer vision, speech processing, and language processing for more than a decade. This has led to a substantial increase in the computation demand at both the server and the edge nodes, which in turn prompted an increase in the computational requirements for specialized hardware for deep learning applications. A myriad of specialized hardware accelerators targeting different deep learning tasks have been on the rise for the better half of the past decade. AI accelerators have also been on the rise on multiple commercial hardware such as the Neural Engine within the Apple System-on-Chips (SoCs), Google's Edge TPU integrated within multiple SoCs designed for the edge and countless more designed for deploying and accelerating deep learning in the Cloud and at the edge with varying degrees of functionality. There is thus a relentless push for newer accelerator architectures that can deliver ever-improving energy efficiency.
This thesis addresses this by proposing hardware architectures which utilise in-network computation to offload some of the deep learning inference and training computation into the Network-on-Chip (NoC) while data is being routed. We tackle this using three different accelerator designs where we explore this design paradigm of infusing compute elements within NoCs which allow for higher energy efficiency for AI-based workloads by being able to perform computation along with the data routing, demonstrating the benefits of \textit{AI accelerators with in-network computing}. The works proposed in this thesis leverage this architectural paradigm to offload computation to these NoCs while also routing the data across different Processing Elements, which effectively reduces additional data movement and possibly also reduces the need for additional compute blocks.
First, the thesis proposes a NoC-based accelerator consisting of heterogeneous Processing Elements each performing Convolution or Matrix Multiplication and connecting them using unique NoCs which integrate computation elements within the NoCs in order to perform some of the inference and training computations while the data is routed between different cores. Next, we utilise the same architectural design methodology to propose a non-linear approximator which can work in conjunction with NoC-based hardware accelerators. The proposed non-linear approximator utilises an approximation methodology which maps non-linear functions used within deep learning models to be approximated using a learnable piecewise linear function, and maps this piecewise linear function to a NoC-based vector unit. This leads to the NoC-based vector unit being much more area and power-efficient than the existing state-of-the-art approximators. Finally, this thesis leads upto a dynamically reconfigurable NoC based accelerator which is able to efficiently map and execute a sparse combination of a Mixture-of-Experts based neural network models, which are used in neural network models for performing multi-task learning using a single model. Putting it all together, the dynamically reconfigurable NoC-based architecture combines all the proposed methodologies within a single accelerator design, which gives it more versatility to map and run diverse neural network models for higher performance and energy efficiency.