Whole-Home Gesture Recognition Using Wireless Signals

Whole-Home Gesture Recognition Using Wireless Signals

September 30-October 4, Miami, FL, USA | Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel
WiSee is a novel gesture recognition system that uses wireless signals (e.g., Wi-Fi) to enable whole-home sensing and recognition of human gestures. Unlike traditional systems that require line-of-sight or body instrumentation, WiSee leverages the ability of wireless signals to pass through walls and not require direct line-of-sight. It achieves this by analyzing the minute Doppler shifts and multi-path distortions in wireless signals caused by human motion. WiSee uses the Doppler shift property, which is the frequency change of a wave as its source moves relative to the observer. By transforming the received signal into a narrowband pulse with a few Hertz bandwidth, WiSee tracks the frequency of this pulse to detect small Doppler shifts from human gestures. WiSee addresses two main challenges: (1) extracting Doppler shifts from wireless signals and (2) distinguishing gestures in the presence of multiple users. For the first challenge, WiSee uses the Doppler shift from human motion to classify gestures. For the second, it leverages MIMO capabilities in 802.11n to focus on gestures from a particular user. WiSee uses a repetitive gesture as a preamble to estimate the MIMO channel that maximizes the energy of the reflections from the user. Once the receiver locks on to this channel, it classifies gestures using Doppler shifts. WiSee was implemented using USRP-N210 hardware and evaluated in both an office environment and a two-bedroom apartment. It successfully classified nine gestures with an average accuracy of 94%. The system performed well in line-of-sight, non-line-of-sight, and through-the-wall scenarios. WiSee's average false positive rate was 2.63 events per hour when using a preamble with two gesture repetitions, and it reduced to 0.07 events per hour with four repetitions. WiSee can also recognize gestures in the presence of multiple users, with an average accuracy of 90% when three other users were present. WiSee's contributions include the first wireless system that enables gesture recognition in line-of-sight, non-line-of-sight, and through-the-wall scenarios. It presents algorithms to extract gesture information from communication-based wireless signals, specifically showing how to extract minute Doppler shifts from wideband OFDM transmissions. Finally, using a proof-of-concept prototype, WiSee demonstrates that it can detect a set of nine whole-body gestures in typical environments. WiSee represents a significant step towards leveraging existing wireless networks to enable novel human-computer interaction mechanisms such as whole-home gesture recognition.WiSee is a novel gesture recognition system that uses wireless signals (e.g., Wi-Fi) to enable whole-home sensing and recognition of human gestures. Unlike traditional systems that require line-of-sight or body instrumentation, WiSee leverages the ability of wireless signals to pass through walls and not require direct line-of-sight. It achieves this by analyzing the minute Doppler shifts and multi-path distortions in wireless signals caused by human motion. WiSee uses the Doppler shift property, which is the frequency change of a wave as its source moves relative to the observer. By transforming the received signal into a narrowband pulse with a few Hertz bandwidth, WiSee tracks the frequency of this pulse to detect small Doppler shifts from human gestures. WiSee addresses two main challenges: (1) extracting Doppler shifts from wireless signals and (2) distinguishing gestures in the presence of multiple users. For the first challenge, WiSee uses the Doppler shift from human motion to classify gestures. For the second, it leverages MIMO capabilities in 802.11n to focus on gestures from a particular user. WiSee uses a repetitive gesture as a preamble to estimate the MIMO channel that maximizes the energy of the reflections from the user. Once the receiver locks on to this channel, it classifies gestures using Doppler shifts. WiSee was implemented using USRP-N210 hardware and evaluated in both an office environment and a two-bedroom apartment. It successfully classified nine gestures with an average accuracy of 94%. The system performed well in line-of-sight, non-line-of-sight, and through-the-wall scenarios. WiSee's average false positive rate was 2.63 events per hour when using a preamble with two gesture repetitions, and it reduced to 0.07 events per hour with four repetitions. WiSee can also recognize gestures in the presence of multiple users, with an average accuracy of 90% when three other users were present. WiSee's contributions include the first wireless system that enables gesture recognition in line-of-sight, non-line-of-sight, and through-the-wall scenarios. It presents algorithms to extract gesture information from communication-based wireless signals, specifically showing how to extract minute Doppler shifts from wideband OFDM transmissions. Finally, using a proof-of-concept prototype, WiSee demonstrates that it can detect a set of nine whole-body gestures in typical environments. WiSee represents a significant step towards leveraging existing wireless networks to enable novel human-computer interaction mechanisms such as whole-home gesture recognition.
Reach us at info@study.space