Algorithm-Hardware Co-Design for Energy-Efficient Neural Network Acceleration
COM3 Level 2
SR21, COM3 02-60
Abstract:
Neural networks are widely used in various real-world applications, ranging from small-scale models with thousands of parameters running on lightweight edge workloads to large-scale models with billions of parameters processing over hundreds of thousands of input values in the cloud. Efficiently running neural networks across different scenarios has become increasingly important. General-purpose hardware, such as GPUs and TPUs, is commonly used to accelerate these workloads. However, it often falls short of meeting diverse requirements due to its inherent lack of scenario-specific optimization. Meanwhile, algorithms cannot achieve optimal performance without considering hardware characteristics. To bridge this gap, optimization from both algorithmic and hardware perspectives presents a promising solution, significantly improving the efficiency of neural network execution in specific scenarios.
This thesis explores algorithm-hardware co-designs for energy-efficient neural network acceleration across various scenarios. For small workloads, spiking neural networks (SNNs) demonstrate superior efficiency due to their high sparsity. To leverage this advantage, we developed a novel and efficient mapping algorithm that enables SNN models to fully utilize the proposed low-power neuromorphic hardware for lightweight tasks such as MNIST. For medium-sized workloads, such as traditional CNNs, achieving high accuracy on large datasets like ImageNet requires large networks, which SNNs struggle to support efficiently. To address this limitation, we drew inspiration from SNNs and propose an event-driven dataflow along with a customized CNN accelerator to efficiently support CNN workloads. With the support of novel dataflow and hardware, MnF achieves high hardware utilization and low memory requirements. For large-scale workloads, such as large language models (LLMs), we introduce a software-hardware co-design that accelerates LLM fine-tuning while incorporating structured sparsity, which is the first work that introduce hardware-friendly structured sparsity into LLM fine-tuning. Furthermore, to analyze the efficiency of different neural network architectures and provide insights into network design under various scenarios, we developed a simulator that integrates energy consumption and latency metrics for SNNs, MLPs, CNNs, and Transformers. Through these contributions, this thesis aims to advance energy-efficient neural network acceleration and provide guidance on optimizing neural networks across diverse scenarios. Compared to state-of-the-art techniques, our solution is more energy efficient. We believe that the approaches proposed in the thesis will serve as a foundation for the future development of energy-efficient neural network acceleration.