Dual adaptive training of photonic neural networks

Artificial intelligence (AI), powered by deep neural networks (DNNs), uses brain-inspired information processing mechanisms to approach human-level performance in complex tasks¹, and has already achieved major applications ranging from translating languages², image recognition³ and cancer diagnosis⁴ to fundamental science⁵. The vast majority of AI algorithms have been implemented via digital electronic computing platforms—such as graphics- and tensor-processing units—to support their major computational requirements; however, the computational performance that AI demands from processors has grown rapidly, greatly exceeding the development of digital electronic computing imposed by Moore’s law and the upper limit of computing energy efficiency^6–8. Constructing photonic neural network (PNN) systems for AI tasks with analogue photonic computing has attracted increasing attention and is expected to be the next-generation AI computing modality due to its advantages of low latency, high bandwidth and low power consumption. The fundamental characteristic of photons and the principle of light–matter interactions (for example, diffraction^9–11 and interference^12–14 based on free-space optics or integrated photonic circuits) have been used to implement various neuromorphic photonic computing architectures such as convolutional neural networks^15–18, spiking neural networks^19–21, recurrent neural networks^22,23 and reservoir computing^24–26.

An effective training approach is one of the most critical aspects for DNNs to learn a model and guarantee high inference accuracy. The DNNs constructed using software on a digital electronic computer are generally trained using the backpropagation algorithm²⁷. Such a training mechanism provides the basis for the in silico training of photonic DNNs, which establishes the PNN models in computers to simulate physical systems, trains models through backpropagation and deploys the trained model parameters to physical systems; however, the inherent systematic errors of analogue computing from different sources (for example, geometric and fabrication errors) cause a deviation between the in silico-trained PNN model and the physical system, resulting in performance degeneration during direct deployment^11,28,29. To address the systematic errors, in situ training approaches (training PNNs on the physical systems with experimental measurements) have drawn increasing attention for optimizing the PNN models for practical applications^11,29–34. Nevertheless, the existing in situ training methods still confront great challenges in training large-scale PNNs with major systematic errors, hindering the construction...