Presentation
Using Probabilistic Model Rollouts to Boost the Sample Efficiency of Reinforcement Learning for Automated Analog Circuit Sizing
SessionAI for Analog Synthesis
DescriptionDespite recent advances in algorithms such as the use of reinforcement learning, analog circuit sizing optimization remains a challenging task that demands numerous circuit simulations, hence extensive CPU times. This paper presents the application of Model-Based Policy Optimization (MBPO) to boost the sample efficiency of reinforcement learning for analog circuit sizing. This method leverages an ensemble of probabilistic dynamic models to generate short rollouts branched from real data for a fast extensive exploration of the design space, thereby speeding up the learning process of the reinforcement learning agent and enhancing its convergence. Integrated in the Twin Delayed DDPG (TD3) algorithm, our new model-based TD3 (MBTD3) approach has been validated on analog circuits of different complexity, outperforming the existing model-free TD3 method by achieving power/area-optimal design solutions with up to 3x fewer simulations and half the run time. In addition, for larger analog circuits, we present a multi-agent version of MBTD3 in which multiple simultaneous agents use global probabilistic models for sizing different blocks within the circuit. Demonstrated for a complex data receiver circuit, it surpasses the model-free multi-agent TD3 method at 2x less simulations and half the run time. These novel methods highly boost the efficiency of automated analog circuit sizing.
Event Type
Research Manuscript
TimeThursday, June 2711:00am - 11:15am PDT
Location3002, 3rd Floor
EDA
Analog CAD, Simulation, Verification and Test