Presentation
Scalable Multi-task Deep Inference on Resource Constrained Energy Harvesting System
DescriptionMulti-task Deep Neural Network (DNN) inference on Energy Harvesting (EH) devices has received limited attention, particularly in scenarios where energy availability fluctuates significantly. This poses substantial implementation challenges, especially in designing DNN architecture that support both varying energy and multi-tasking on resource-limited systems.Therefore, this work presents a comprehensive software/hardware co-design framework that introduces a Unified DNN model.
This Unified model has the flexibility to scale the network depending on the varying environment while performing multiple correlated tasks at the same time. To achieve such model, we propose inter-task and intra-task shared-weight design approach, where the inter-task design unifies the commonality of the extracted features, and the intra-task design unifies the shared information between multi-level sparsity.
We further present an on-device efficient implementation scheme where the Unified model is compressed with the promise of consistent inference accuracy. A runtime weight recollection process is also presented that guarantees dynamic DNN scalability.
Experimental results show that our multi-task Unified DNN, with proposed hardware implementation architecture can enable up to 1.8x inference latency reduction, 48% energy efficiency, and 3.5x memory conservation.
This Unified model has the flexibility to scale the network depending on the varying environment while performing multiple correlated tasks at the same time. To achieve such model, we propose inter-task and intra-task shared-weight design approach, where the inter-task design unifies the commonality of the extracted features, and the intra-task design unifies the shared information between multi-level sparsity.
We further present an on-device efficient implementation scheme where the Unified model is compressed with the promise of consistent inference accuracy. A runtime weight recollection process is also presented that guarantees dynamic DNN scalability.
Experimental results show that our multi-task Unified DNN, with proposed hardware implementation architecture can enable up to 1.8x inference latency reduction, 48% energy efficiency, and 3.5x memory conservation.
Event Type
Work-in-Progress Poster
TimeTuesday, June 256:00pm - 7:00pm PDT
LocationLevel 2 Lobby
AI
Autonomous Systems
Cloud
Design
EDA
Embedded Systems
IP
Security