Presentation
Understanding the Upper Bounds of Energy Efficiency in a Computing-in-Memory Processor and How to Approach the Limit
DescriptionThe computing-in-memory (CIM) architecture has demonstrated high energy efficiency on memory-intensive and computation-intensive AI workloads. Despite the high energy efficiency of CIM circuits (i.e. macros), there remains a significant gap between the macro-level and processor-level energy efficiency in existing CIM chips. The key bottleneck is that, non-trivial surrounding modules are still necessary to implement an end-to-end CIM processor, including the SRAMs, register buffers, accumulators, etc. These surrounding modules show varied influences under different CIM configurations and workloads.
This work is motivated to explore the upper bound of energy efficiency in a CIM processor, and explore methods to approach the theoretical limit. The main contributions of this paper include: 1) Reveal the necessary modules of a CIM processor for different scales of applications; 2) Propose a quantitative analysis of the processor-level energy efficiency for different CIM architectures, as well as the gap between actual values and upper bounds; 3) Indicate the design principles to approach the theoretical upper bound of energy efficiency. Experiment results show an obviously varied processor/macro efficiency ratio (9.06%-44.3%) under varied design parameters and AI workloads. With the proposed flexible parallelism, the processor/macro efficiency ratio can be improved by up to 15.0%.
This work is motivated to explore the upper bound of energy efficiency in a CIM processor, and explore methods to approach the theoretical limit. The main contributions of this paper include: 1) Reveal the necessary modules of a CIM processor for different scales of applications; 2) Propose a quantitative analysis of the processor-level energy efficiency for different CIM architectures, as well as the gap between actual values and upper bounds; 3) Indicate the design principles to approach the theoretical upper bound of energy efficiency. Experiment results show an obviously varied processor/macro efficiency ratio (9.06%-44.3%) under varied design parameters and AI workloads. With the proposed flexible parallelism, the processor/macro efficiency ratio can be improved by up to 15.0%.
Event Type
Work-in-Progress Poster
TimeTuesday, June 256:00pm - 7:00pm PDT
LocationLevel 2 Lobby
AI
Autonomous Systems
Cloud
Design
EDA
Embedded Systems
IP
Security