Presentation
Zeroth-Order Optimization of Optical Neural Networks with Linear Combination Natural Gradient and Calibrated Model
DescriptionOptical neural networks (ONNs) have attracted great attention due to their low energy consumption and high-speed processing. The usual neural network training scheme leads to poor performance for ONNs because of their special parameterization and fabrication variations. This paper contributes to extend zeroth-order (ZO) optimization, which can be used to train such ONNs, in two ways. The first is to propose linear combination natural gradient, which mitigates the optimization difficulty caused by the special parameterization of an ONN. The second is to generate a guided direction vector by calibration for better guessing than random vectors generated in ZO optimization. Experimental results show that the two extensions significantly outperformed the existing ZO optimization and related methods with little computational overhead.
Event Type
Research Manuscript
TimeWednesday, June 2610:30am - 10:45am PDT
Location3001, 3rd Floor
AI
AI/ML Algorithms