Presentation
AdaP-CIM: Compute-in-Memory Based Neural Network Accelerator using Adaptive Posit and Speculative Alignment
DescriptionLarge neural networks, especially transformer-based models, present two critical challenges that exacerbate the memory wall issue in AI accelerator designs. First, the increased dynamic range of the weights requires higher precision quantization formats, leading to higher memory capacity requirements. Second, the exponential growth in model parameters incurs more data movement, leading to increased latency and power consumption. In this study, we propose two novel approaches to address these problems. First, based on Posit, we introduce a new format called adaptive Posit (AdaP), which dynamically extends the dynamic range of its representation at run time with minimal hardware overhead. AdaP, utilizing two exponent encoding schemes, accommodates the data distribution with lower quantization error compared to regular Posit. Second, we propose to use compute-in-memory (CIM) architecture to implement AdaP multiply-and-accumulate (MAC) computation to reduce weight data movement. Traditional CIM proposed for floating-point-alike MAC computation uses a comparator tree (CT) to compute the maximum exponent, enabling the CIM to focus on integer MAC. However, the CT-based design has poor scalability as the number of inputs increases. To address this, we propose a speculative input alignment design that significantly reduces the delay, area, and power consumption for the max exponent computation. Software evaluations show that 8-bit AdaP incurs a negligible 0.25% F1 score reduction on the XLM language identification benchmark compared to the full-precision baseline. Hardware synthesis and simulation results further illustrate that our approach achieves 55% energy efficiency and 2.4x area efficiency improvement compared to the state-of-the-art posit processing element.
Event Type
Work-in-Progress Poster
TimeWednesday, June 265:00pm - 6:00pm PDT
LocationLevel 2 Lobby
AI
Autonomous Systems
Cloud
Design
EDA
Embedded Systems
IP
Security