Presentation

· Contributors · Organizations · Search Program · Flagged · Happening Now

Deep Reorganization: Retaining Residuals in TinyML

SessionDo-More-with-Less: Optimizing AI Models for Inference Efficiencies

DescriptionDesigning intelligent, tiny devices with limited memory is immensely challenging, exacerbated by the additional memory requirement of residual connections in deep neural networks. In contrast to existing approaches that eliminate residuals to reduce peak memory usage at the cost of significant accuracy degradation, this paper presents DERO, which reorganizes residual connections by leveraging insights into the types and interdependencies of operations across residual connections. Evaluations were conducted across diverse model architectures designed for common computer vision applications. DERO consistently achieves peak memory usage comparable to plain-style models without residuals, while maintaining the accuracy of the original models with residuals.

Authors

National Taiwan Normal University

Ming-Syan Chen

National Taiwan University

Pi-Cheng Hsiu

Academia Sinica

Event Type