
- Opto-Electronic Advances
- Vol. 6, Issue 11, 230140 (2023)
Abstract
Introduction
The deep neural network has developed rapidly and achieved record-breaking performance in a broad range of applications, including computer vision, natural language processing, and other fields. These applications produce huge amounts of data that need to be processed, which calls for advanced processors with high speed, high throughput and low latency. While Moore’s Law approaches saturation and conventional digital computers face the von-Neumann bottleneck due to the separation of memory and processor units. Neuromorphic computing, which emulates the structure and mechanism of the brain, has emerged as a competitive computation paradigm in the post-Moore era
In addition to the advancements in neuromorphic hardware, novel neural network models have also attracted significant attention. In contrast to traditional non-spiking artificial neural networks (ANNs), spiking neural networks (SNNs) have been proposed and designed to enable low power consumption computing on neuromorphic hardware by using discrete spike signals
Spiking neurons serve as the fundamental units of SNNs, and are connected through plastic synapses. In an SNN, the spiking neuron performs the nonlinear spike activation, while the synapse performs the linear weighting function. For the majority of reported photonic neural networks, only linear operation was realized in the photonics domain
Tremendous efforts have been made to realize photonic spiking neurons, aiming to enable all-optical SNNs without the need for frequent OE/EO and AD/DA conversions. Among various simplified spiking neuron models in computational neuroscience, the leaky integrate-and-fire (LIF) model has gained popularity due to its simplicity. Optical implementations of the LIF neuron have received considerable attention
Here, we proposed, fabricated and demonstrated a photonic neuro-synaptic chip that enables simultaneous implementation of both linear weighting and nonlinear spike activation based on a photonic integrated DFB laser with an intracavity saturable absorber (DFB-SA) for the first time. By precisely tuning the external optical injection and the time-varying bias current of the gain region of the DFB-SA laser, we experimentally demonstrate both the intrinsic excitability plasticity and synaptic plasticity in the same DFB-SA laser. The nonlinear spike activation and linear weighting can be simultaneously realized in a single DFB-SA laser, which is the main advantage of the proposed photonic neuro-synaptic chip. In addition, parallel linear weighting and nonlinear spike activation were demonstrated in a constructed prototypical neuromorphic photonics system consisting of two pre-synaptic DFB-SA lasers and one post-synaptic FP-SA laser. To demonstrate the scalability of the DFB-SA laser, a four-channel photonic neuro-synaptic array was further fabricated and applied to perform dot product between two vectors. Through time-multiplexing matrix convolution, we successfully demonstrated pattern classification tasks. We further developed a theoretical model to numerically simulate the photonic integrated DFB-SA laser, validating our experimental findings. The integration of linear weighting and nonlinear spike activation in a single chip opens up new possibilities for efficient and scalable photonic neuromorphic computing, presenting a promising building block for constructing multi-layer photonic SNN hardware within the Indium Phosphide (InP) integration platform.
Experimental setup and method
Principle for photonic neuro-synaptic core
In a biological neural network, the neurons are connected by plastic synapses. As depicted in
Figure 1.
We further designed and fabricated a four-channel photonic neuro-synaptic array to implement two-dimensional (2D) convolution for spiking convolutional neural networks (CNNs). To fully leverage the temporal dynamics of the SNN, we have demonstrated time-multiplexing matrix convolution. The fabricated four-channel DFB-SA laser array and the compact packaged module are depicted in
Model of the DFB-SA laser
In order to gain a deeper understanding of the intrinsic excitability plasticity and synaptic plasticity in the DFB-SA laser, we have developed a comprehensive model of the DFB-SA laser based on the time-dependent coupled-wave equations. We modified the model to incorporate the gain region, SA region, and an external optical injection term
Results
Intrinsic neuron excitability plasticity
The experimental setup for emulating the neuronal intrinsic plasticity and linear weighting using a DFB-SA laser is presented in
Figure 2.
For a LIF spiking neuron, temporal integration, spike threshold and generation as well as the refractory period are crucial information processing mechanisms. Here, we successfully demonstrated these spike-based nonlinear processing mechanisms using the fabricated DFB-SA laser chip. As shown in
Similarly, we fixed the external stimulus strength, and designed 10 pulse pairs with increasing inter-spike interval (ISI) to explore the intrinsic plasticity of the refractory period and temporal integration behavior. Here, the ISI for each input pulse pair is 0.4 ns, 0.6 ns, 0.8 ns, 1 ns, 1.2 ns, 1.4 ns, 1.6 ns, 1.8 ns, 2 ns, and 2.2 ns, respectively. As presented in
Note, the demonstrated intrinsic neuron excitability plasticity implies that, when developing a supervised training algorithm for a photonic SNN consisting of the DFB-SA laser, not only the weights can be considered as adjustable parameters, but the excitability threshold of the DFB-SA laser can also be trainable to accelerate the training process. Thus, a novel hardware-aware training algorithm that combines both the weight and excitability threshold modulation is highly desirable to enhance the performance of photonic SNNs.
Continuously tunable synaptic plasticity
Next, we experimentally demonstrated the linear weighting function of the fabricated DFB-SA laser chip by employing a statically-varying bias current. We defined a periodic optical pulse train as the external stimulus and adjusted the gain current, ensuring that the DFB-SA laser operated in an LIF-like response manner. As illustrated in
Figure 3.
By simply tuning the bias current of the gain region of the DFB-SA laser, we successfully achieved precise control over the amplitude of the weighted optical spikes, thus enabling a photonic integrated spiking neuron with incorporated spike weighting capability. Note, in comparison to biological counterparts that operate on millisecond timescales, the photonic neuro-synaptic unit utilizing the fabricated DFB-SA laser offers significantly faster operation speed in the sub-nanosecond range, thanks to the short carrier lifetime inherent to these chips. Therefore, the proposed photonic neuron-synaptic unit avoids the use of additional photonic weighting elements, making it highly desirable for the further scalability of photonic SNN hardware.
Simultaneous implementation of nonlinear spike activation and linear weighting
In the following, we considered the scenario where a dynamically time-varying gain current was applied to the DFB-SA laser, allowing for the simultaneous implementation of nonlinear spike activation and linear weighting. Note, the nonlinear spike activation is implemented due to the excitability threshold modulation. For the purpose of comparison, we employed a time-varying optical input stimulus and considered different constant gain currents, as depicted in
Figure 4.
Then, we considered a fixed input strength and applied a dynamically modulated gain current. In this experiment, the repetition rate of the input pulse train was set as 0.5 GHz. As presented in
We also considered other input spike trains with higher repetition rates, and found that the linear weighting function was degraded. Some spikes cannot be normally weighted, which is mainly limited by the refractory period. Note, the spike weighting rate can be increased when a larger static bias current is utilized.
We further numerically reproduced the linear weighting function and nonlinear spike activation. The numerical results obtained from the model are presented in the Supplement material Note 3. These results demonstrate that different gain currents result in distinct excitability thresholds. In addition, the modulation of gain current produces weighted spikes with distinguishable amplitude levels. Namely, the numerical results agree well with the experimental findings, thus providing a computational model that facilitates hardware-algorithm co-design and optimization of photonic SNNs utilizing the DFB-SA lasers.
Fully-functional photonic neuromorphic prototypical system
We further constructed a fully-functional prototypical photonic neuromorphic system utilizing the fabricated DFB-SAs for parallel linear weighting and nonlinear spike activation. Note, as the packaged DFB-SAs operate at different wavelengths, we adopted an FP-SA as the post-synaptic neuron because it can support wide bandwidth operation
Figure 5.
The output of the FP-SA reveals that it can yield either 0100 or 1111, depending on the weighting conditions of the DFB-SAs or the excitability threshold of the FP-SA. Consequently, the constructed neuromorphic prototypical system can be flexibly reconfigured to execute spike-based AND as well as OR operations. Note, both the weighted addition operation and nonlinear spike activation are achieved within this photonic neuromorphic prototypical system based on the DFB-SAs and FP-SA, which can be integrated onto the same InP integration platform.
Time multiplexing matrix convolution with four-channel DFB-SA laser array
To demonstrate the scalability of the proposed photonic neuro-synaptic unit based on the DFB-SA, we fabricated a four-channel DFB-SA array and applied it to perform matrix convolution for a convolutional SNN. The threshold for each DFB-SA laser is about 86 mA when VSA=0 V. The measured optical spectra for the four-channel DFB-SA laser array is presented in Supplementary information Section 5. The operation principle is illustrated in
Figure 6.
The spiking CNN was trained using a digital computer to classify the MNIST dataset
We further conducted numerical simulations to explore time-multiplexing parallel dot product using our developed time-dependent traveling wave model. The numerical results of four-channel parallel linear weighting are presented in Supplementary information Section 8. The simulations revealed that the linear weighting operation could be effectively realized in parallel using this configuration. Moreover, we considered a larger convolution kernel size and simulated the case of a 3×3 convolution kernel. The numerical results of 9-channel parallel linear weighting are provided in Supplementary information Section 9. It indicates that, 9-channel parallel dot product could be achieved by employing an array of nine DFB-SAs as the photonic dot product core. We further consider a spiking CNN network similar to the one shown in
Discussions and conclusions
The computation speed of the linear weighting operation in the fabricated four-channel DFB-SA laser array, with statically configured weights, can be estimated at 2×4×10 G=80 GOPS. On the other hand, the nonlinear computation speed for a single channel is approximately 2 G Spike/s, limited by the refractory period. The energy consumption per spike is approximately 19.99 pJ. The area occupied by a single DFB-SA laser chip is around 1500 μm ×300 μm=0.45 mm2. It is important to note that there is significant potential for improving these metrics. Further optimization can be achieved by reducing the threshold of the DFB-SA laser to around or below 5 mA, and by reducing the area of a single DFB-SA laser chip to 300 μm×127 μm. The reconfigurable rate, which is associated with the speed of dynamics weight update, is estimated at 0.5 GHz, significantly faster than that of the thermos-optic phase shifters based on silicon photonics. Moreover, the dynamical weighting speed can be further increased to approximately 10 GHz by reducing the cavity length of the DFB-SA laser to 300 μm, which is compatible with the time-multiplexing matrix convolution requiring fast weight modulation. Additionally, a previously demonstrated 60-channel DFB laser array with high wavelength precision
Compared to silicon photonics-based weighting elements, the proposed DFB-SA laser offers several advantages. Firstly, it eliminates the loss issue as gain amplification can be easily achieved in the InP integration platform. This characteristic is particularly beneficial for the implementation of multi-layer photonic SNNs. On the other hand, weight control using thermo-optic phase shifters in the silicon photonics platform is limited to a tuning rate in the kHz range. In contrast to the weighting devices based on the VCSEL biased below the lasing threshold
In conclusion, we have proposed and successfully fabricated a novel photonic spiking neuron chip based on an integrated DFB-SA laser, which enables simultaneous spike activation and linear weighting functions for the first time. This chip has a simple structure and can be readily integrated on a large scale using commercially mature fabrication processes available in photonics foundries. By adjusting the gain current applied to the DFB-SA laser, the chip can be flexibly reconfigured to function as a linear weighting device or nonlinear spike activation device. The fully-functional neuromorphic prototypical system, comprising DFB-SA lasers and FP-SA laser, successfully performed spike-based AND as well as OR operations. Furthermore, using a fabricated four-channel DFB-SA laser array, we benchmarked the hardware-software collaborative inference on the MNIST dataset, achieving an inference accuracy of 87%. Overall, our work demonstrates the potential of the integrated DFB-SA laser for advancing the field of photonic neuromorphic computing, offering scalability of fully-functional integration of lossless multilayer or deep photonic SNN in a single integrated chip.
References
[1] G Indiveri, SC Liu. Memory and information processing in neuromorphic systems. Proc IEEE, 1379-1397(2015).
[2] K Roy, A Jaiswal, P Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 607-617(2019).
[3] D Marković, A Mizrahi, D Querlioz, J Grollier. Physics for neuromorphic computing. Nat Rev Phys, 499-510(2020).
[4] RA Nawrocki, RM Voyles, SE Shaheen. A mini review of neuromorphic architectures and implementations. IEEE Trans Electron Devices, 3819-3829(2016).
[5] CD Schuman, TE Potok, RM Patton, JD Birdwell, ME Dean et al. A survey of neuromorphic computing and neural networks in hardware. arXiv preprint arXiv: 1705.06963(2017).
[6] E Painkras, LA Plana, J Garside, S Temple, F Galluppi et al. SpiNNaker: a 1-W 18-core system-on-chip for massively-parallel neural network simulation. IEEE J Solid-State Circuits, 1943-1953(2013).
[7] BV Benjamin, PR Gao, E McQuinn, S Choudhary, AR Chandrasekaran et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc IEEE, 699-716(2014).
[8] PA Merolla, JV Arthur, R Alvarez-Icaza, AS Cassidy, J Sawada et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 668-673(2014).
[9] JC Shen, D Ma, ZH Gu, M Zhang, XL Zhu et al. Darwin: a neuromorphic hardware co-processor based on spiking neural networks. Sci China Inform Sci, 1-5(2016).
[10] M Davies, N Srinivasa, TH Lin, G Chinya, YQ Cao et al. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro, 82-99(2018).
[11] J Pei, L Deng, S Song, MG Zhao, YH Zhang et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 106-111(2019).
[12] G Wetzstein, A Ozcan, S Gigan, SH Fan, D Englund et al. Inference in artificial intelligence with deep optics and photonics. Nature, 39-47(2020).
[13] BJ Shastri, AN Tait, de Lima T Ferreira, WHP Pernice, H Bhaskaran et al. Photonics for artificial intelligence and neuromorphic computing. Nat Photonics, 102-114(2021).
[14] HL Zhou, JJ Dong, JW Cheng, WC Dong, CR Huang et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci Appl, 30(2022).
[15] CR Huang, VJ Sorger, M Miscuglio, M Al-Qadasi, A Mukherjee et al. Prospects and applications of photonic neural networks. Adv Phys X, 1981155(2022).
[16] HX Qi, ZC Du, XY Hu, JY Yang, SS Chu et al. High performance integrated photonic circuit based on inverse design method. Opto-Electron Adv, 210061(2022).
[17] CH Li, W Du, YX Huang, JH Zou, LZ Luo et al. Photonic synapses with ultralow energy consumption for artificial visual perception and brain storage. Opto-Electron Adv, 210069(2022).
[18] SM Jiao, JW Liu, LW Zhang, FH Yu, GM Zuo et al. All-optical logic gate computing for high-speed parallel information processing. Opto-Electron Sci, 220010(2022).
[19] XY Meng, GJ Zhang, NN Shi, GY Li, J Azaña et al. Compact optical convolution processing unit based on multimode interference. Nat Commun, 3000(2023).
[20] F Zhang, YH Guo, MB Pu, LW Chen, MF Xu et al. Meta-optics empowered vector visual cryptography for high security and rapid decryption. Nat Commun, 1946(2023).
[21] C He, D Zhao, F Fan, HQ Zhou, X Li et al. Pluggable multitask diffractive neural networks based on cascaded metasurfaces. Opto-Electron Adv, 230005(2024).
[22] W Maass. Networks of spiking neurons: the third generation of neural network models. Neural Netw, 1659-1671(1997).
[23] R Gütig, H Sompolinsky. The tempotron: a neuron that learns spike timing–based decisions. Nat Neurosci, 420-428(2006).
[24] F Ponulak, A Kasiński. Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting. Neural Comput, 467-510(2010).
[25] J Feldmann, N Youngblood, CD Wright, H Bhaskaran, WHP Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 208-214(2019).
[26] SY Xiang, ZX Ren, ZW Song, YH Zhang, XX Guo et al. Computing primitive of fully VCSEL-based all-optical spiking neural network for supervised learning and pattern classification. IEEE Trans Neural Netw Learn Syst, 2494-2505(2021).
[27] A Jha, CR Huang, HT Peng, B Shastri, PR Prucnal. Photonic spiking neural networks and graphene-on-silicon spiking neurons. J Lightwave Technol, 2901-2914(2022).
[28] SY Xiang, YC Shi, XX Guo, YH Zhang, HJ Wang et al. Hardware-algorithm collaborative computing with photonic spiking neuron chip based on an integrated Fabry–Perot laser with a saturable absorber. Optica, 162-171(2023).
[29] AN Tait, de Lima T Ferreira, E Zhou, AX Wu, MA Nahmias et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci Rep, 7430(2017).
[30] YC Shen, NC Harris, S Skirlo, M Prabhu, T Baehr-Jones et al. Deep learning with coherent nanophotonic circuits. Nat Photonics, 441-446(2017).
[31] ZG Cheng, C Ríos, WHP Pernice, CD Wright, H Bhaskaran. On-chip photonic synapse. Sci Adv, e1700160(2017).
[32] HL Zhou, YH Zhao, X Wang, DS Gao, JJ Dong et al. Self-configuring and reconfigurable silicon photonic signal processor. ACS Photonics, 792-799(2020).
[33] J Feldmann, N Youngblood, M Karpov, H Gehring, X Li et al. Parallel convolutional processing using an integrated photonic tensor core. Nature, 52-58(2021).
[34] XY Xu, MX Tan, B Corcoran, JY Wu, A Boes et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature, 44-51(2021).
[35] SF Xu, J Wang, HW Shu, ZK Zhang, SC Yi et al. Optical coherent dot-product chip for sophisticated deep learning regression. Light Sci Appl, 221(2021).
[36] H Zhang, M Gu, XD Jiang, J Thompson, H Cai et al. An optical neural chip for implementing complex-valued neural network. Nat Commun, 457(2021).
[37] XH Guo, JL Xiang, YJ Zhang, YK Su. Integrated neuromorphic photonics: synapses, neurons, and neural networks. Adv Photonics Res, 2000212(2021).
[38] JW Cheng, YH Zhao, WK Zhang, HL Zhou, DM Huang et al. A small microring array that performs large complex-valued matrix-vector multiplication. Front Optoelectron, 15(2022).
[39] PR Prucnal, BJ Shastri, de Lima T Ferreira, MA Nahmias, AN Tait. Recent progress in semiconductor excitable lasers for photonic spike processing. Adv Opt Photonics, 228-299(2016).
[40] J Robertson, E Wade, Y Kopp, J Bueno, A Hurtado. Toward neuromorphic photonic networks of ultrafast spiking laser neurons. IEEE J Sel Top Quantum Electron, 7700715(2020).
[41] YH Zhang, J Robertson, SY Xiang, M Hejda, J Bueno et al. All-optical neuromorphic binary convolution with a spiking VCSEL neuron for image gradient magnitudes. Photonics Res, B201-B209(2021).
[42] MA Nahmias, BJ Shastri, AN Tait, PR Prucnal. A leaky integrate-and-fire laser neuron for ultrafast cognitive computing. IEEE J Sel Top Quantum Electron, 1800212(2013).
[43] BJ Shastri, MA Nahmias, AN Tait, AW Rodriguez, B Wu et al. Spike processing with a graphene excitable laser. Sci Rep, 19126(2016).
[44] I Chakraborty, G Saha, G Sengupta, K Roy. Toward fast neural computing using all-photonic phase change spiking neurons. Sci Rep, 12980(2018).
[45] F Selmi, R Braive, G Beaudoin, I Sagnes, R Kuszelewicz et al. Relative refractory period in an excitable semiconductor laser. Phys Rev Lett, 183902(2014).
[46] HT Peng, G Angelatos, de Lima T Ferreira, MA Nahmias, AN Tait et al. Temporal information processing with an integrated laser neuron. IEEE J Sel Top Quantum Electron, 5100209(2020).
[47] JL Xiang, YJ Zhang, YT Zhao, XH Guo, YK Su. All-optical silicon microring spiking neuron. Photonics Res, 939-946(2022).
[48] DZ Zheng, SY Xiang, XX Guo, YH Zhang, BL Gu et al. Experimental demonstration of coherent photonic neural computing based on a Fabry–Perot laser with a saturable absorber. Photonics Res, 65-71(2023).
[49] YC Shi, SM Li, XF Chen, LY Li, JS Li et al. High channel count and high precision channel spacing multi-wavelength laser array for future PICs. Sci Rep, 7377(2014).
[50] YC Shi, SY Xiang, XX Guo, YH Zhang, HJ Wang et al. Photonic integrated spiking neuron chip based on a self-pulsating DFB laser with a saturable absorber. Photonics Res, 1382-1389(2023).
[51] H Beck, Y Yaari. Plasticity of intrinsic neuronal properties in CNS disorders. Nat Rev Neurosci, 357-369(2008).
[53] JA Alanis, J Robertson, M Hejda, A Hurtado. Weight adjustable photonic synapse by nonlinear gain in a vertical cavity semiconductor optical amplifier. Appl Phys Lett, 201104(2021).
[54] J Robertson, JA Alanis, M Hejda, A Hurtado. Photonic synaptic system for MAC operations by interconnected vertical cavity surface emitting lasers. Opt Mater. Express, 1417-1426(2022).

Set citation alerts for the article
Please enter your email address