An artificial visual neuron with multiplexed rate and time-to-first-spike coding

An artificial visual neuron with multiplexed rate and time-to-first-spike coding

01 May 2024 | Fanfan Li, Dingwei Li, Chuanqing Wang, Guolei Liu, Rui Wang, Huihui Ren, Yingjie Tang, Yan Wang, Yitong Chen, Kun Liang, Qi Huang, Mohamad Sawan, Min Qiu, Hong Wang, Bowen Zhu
This paper presents an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The neuron integrates an In₂O₃ synaptic phototransistor and an NbOₓ Mott memristor, mimicking biological photoreceptors and retinal ganglion neurons, respectively. The RTF coding scheme combines rate coding, which encodes visual information at different spiking frequencies, with time-to-first-spike (TTFS) coding, which provides precise and energy-efficient temporal information. This multiplexed coding scheme enhances the computing capability and efficacy of artificial visual neurons in spiking neural networks (SNNs). The hardware-based SNN with RTF coding demonstrates good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The results demonstrate the feasibility of developing highly efficient spike-based neuromorphic hardware.This paper presents an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The neuron integrates an In₂O₃ synaptic phototransistor and an NbOₓ Mott memristor, mimicking biological photoreceptors and retinal ganglion neurons, respectively. The RTF coding scheme combines rate coding, which encodes visual information at different spiking frequencies, with time-to-first-spike (TTFS) coding, which provides precise and energy-efficient temporal information. This multiplexed coding scheme enhances the computing capability and efficacy of artificial visual neurons in spiking neural networks (SNNs). The hardware-based SNN with RTF coding demonstrates good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The results demonstrate the feasibility of developing highly efficient spike-based neuromorphic hardware.
Reach us at info@study.space
[slides and audio] An artificial visual neuron with multiplexed rate and time-to-first-spike coding.