16 April 2024 | Pengzhan Li, Mingzhen Zhang, Qingli Zhou, Qinghua Zhang, Donggang Xie, Ge Li, Zhuohui Liu, Zheng Wang, Erjia Guo, Meng He, Can Wang, Lin Gu, Guozhen Yang, Kuijuan Jin & Chen Ge
A reconfigurable optoelectronic transistor is developed for multimodal recognition, capable of performing both physical reservoir and synaptic functions. The device exhibits tunable time-scales under optical and electrical stimuli, with nonlinear volatile properties suitable for reservoir computing and multimodal pre-processing. The non-volatility and programmability of the device, achieved through ion insertion/extraction via electrolyte gating, are verified. The device's superior performance in mimicking human perception of dynamic and static multisensory information is demonstrated. The study provides an exciting paradigm for multimodal reconfigurable devices and opens avenues for mimicking biological multisensory fusion.
The human brain integrates sensory information for high-level cognition, while traditional computers face challenges in processing dynamic information due to high latency and energy consumption. Biologically-inspired artificial systems are expected to solve this, with a promising approach being the development of compact parallel optoelectronic fusion hardware systems.
The multimodal optoelectronic system includes a reservoir for pre-processing and an artificial neural network (ANN) for post-processing. The reservoir requires volatility, while the ANN requires non-volatility, making it challenging to integrate both in a single device. The study presents a BSO-EGT that integrates multimodal sensing, memory, and processing functions, capable of emulating short- and long-term plasticity under optical and electrical stimulation. The device's reconfigurable properties allow it to function as both a reservoir and an ANN, enabling multimodal recognition.
The BSO-EGT exhibits tunable temporal dynamics under optical stimulation, with a double exponential decay model describing the current response. The device's response to electrical stimuli shows reconfigurable dynamics, with the ability to switch between volatile and non-volatile modes. The device's multimodal characteristics allow it to process both optical and electrical inputs, enabling the recognition of multimodal information.
The BSO-EGT was tested for static image recognition, demonstrating high accuracy in recognizing contaminated images. The device's ability to process both optical and electrical inputs allows it to recognize images with high accuracy, even when parts of the image are invisible. The device was also tested for dynamic gesture recognition, showing high accuracy in recognizing gestures when combined with audio-visual inputs.
The study also demonstrates the potential of the BSO-EGT for wide application wavelength, using IGZO as an alternative channel material. The IGZO-EGT shows similar reconfigurable properties and can be used for multimodal recognition tasks. The study highlights the potential of the BSO-EGT for neuromorphic applications, with the ability to process dynamic information and simulate biological multisensory fusion. The device's performance in multimodal recognition tasks demonstrates its potential for advanced neuromorphic applications.A reconfigurable optoelectronic transistor is developed for multimodal recognition, capable of performing both physical reservoir and synaptic functions. The device exhibits tunable time-scales under optical and electrical stimuli, with nonlinear volatile properties suitable for reservoir computing and multimodal pre-processing. The non-volatility and programmability of the device, achieved through ion insertion/extraction via electrolyte gating, are verified. The device's superior performance in mimicking human perception of dynamic and static multisensory information is demonstrated. The study provides an exciting paradigm for multimodal reconfigurable devices and opens avenues for mimicking biological multisensory fusion.
The human brain integrates sensory information for high-level cognition, while traditional computers face challenges in processing dynamic information due to high latency and energy consumption. Biologically-inspired artificial systems are expected to solve this, with a promising approach being the development of compact parallel optoelectronic fusion hardware systems.
The multimodal optoelectronic system includes a reservoir for pre-processing and an artificial neural network (ANN) for post-processing. The reservoir requires volatility, while the ANN requires non-volatility, making it challenging to integrate both in a single device. The study presents a BSO-EGT that integrates multimodal sensing, memory, and processing functions, capable of emulating short- and long-term plasticity under optical and electrical stimulation. The device's reconfigurable properties allow it to function as both a reservoir and an ANN, enabling multimodal recognition.
The BSO-EGT exhibits tunable temporal dynamics under optical stimulation, with a double exponential decay model describing the current response. The device's response to electrical stimuli shows reconfigurable dynamics, with the ability to switch between volatile and non-volatile modes. The device's multimodal characteristics allow it to process both optical and electrical inputs, enabling the recognition of multimodal information.
The BSO-EGT was tested for static image recognition, demonstrating high accuracy in recognizing contaminated images. The device's ability to process both optical and electrical inputs allows it to recognize images with high accuracy, even when parts of the image are invisible. The device was also tested for dynamic gesture recognition, showing high accuracy in recognizing gestures when combined with audio-visual inputs.
The study also demonstrates the potential of the BSO-EGT for wide application wavelength, using IGZO as an alternative channel material. The IGZO-EGT shows similar reconfigurable properties and can be used for multimodal recognition tasks. The study highlights the potential of the BSO-EGT for neuromorphic applications, with the ability to process dynamic information and simulate biological multisensory fusion. The device's performance in multimodal recognition tasks demonstrates its potential for advanced neuromorphic applications.