Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface

Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface

15 January 2024 | Jin Pyo Lee, Hanhyeok Jang, Yeonwoo Jang, Hyeonseo Song, Suwoo Lee, Pooi See Lee & Jiyun Kim
The paper presents a personalized skin-integrated facial interface (PSiFI) system designed to recognize multi-modal emotional information, combining verbal and non-verbal expressions. The PSiFI is self-powered, stretchable, transparent, and features bidirectional triboelectric strain and vibration sensors, enabling real-time emotion recognition. The system includes a data processing circuit for wireless data transfer and a convolutional neural network (CNN) for classification. The device is fully integrated with machine learning techniques, allowing for accurate emotion recognition even when wearing a mask. The system demonstrates applications in VR environments, where it can provide digital concierge services based on users' emotional states. The study highlights the potential of PSiFI in enhancing human-machine interaction and digital transformation.The paper presents a personalized skin-integrated facial interface (PSiFI) system designed to recognize multi-modal emotional information, combining verbal and non-verbal expressions. The PSiFI is self-powered, stretchable, transparent, and features bidirectional triboelectric strain and vibration sensors, enabling real-time emotion recognition. The system includes a data processing circuit for wireless data transfer and a convolutional neural network (CNN) for classification. The device is fully integrated with machine learning techniques, allowing for accurate emotion recognition even when wearing a mask. The system demonstrates applications in VR environments, where it can provide digital concierge services based on users' emotional states. The study highlights the potential of PSiFI in enhancing human-machine interaction and digital transformation.
Reach us at info@study.space
[slides and audio] Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface