Please login first
Hand Gesture to Sound: A Real-Time DSP-Based Audio Modulation System for Assistive Interaction
* 1 , 2 , 1 , 1 , 1
1  Department of Electronic Engineering, NED University of Engineering and Technology, Karachi, Pakistan
2  Department of Telecommunications Engineering, NED University of Engineering and Technology, Karachi, Pakistan
Academic Editor: Francisco Falcone

https://doi.org/10.3390/ECSA-12-26516 (registering DOI)
Abstract:

This paper presents the design, development, and evaluation of an embedded hardware and digital signal processing (DSP) based real-time gesture-controlled system. The system architecture utilizes an MPU6050 inertial measurement unit (IMU), Arduino Uno micro-controller, and Python-based audio interface to recognize and classify directional hand gestures, and transform them into auditory commands. Wrist tilts, i.e., left, right, forward, and backward, are recognized using a hybrid algorithm that uses thresholding, moving average filtering, and low-pass smoothing to remove sensor noise and transient errors. Hardware setup utilizes I2C-based sensor acquisition, onboard preprocessing on Arduino, and serial communication with a host computer running a Python script to trigger audio playing using the playsound library. Four gestures are programmed for basic needs: Hy-dration Request, Meal Support, Restroom Support, and Emergency Alarm. Experimental evaluation, conducted over more than 50 iterations per gesture in controlled laboratory setup, resulted in a mean recognition rate of 92%, with system latency of 120 to 150 milli-seconds. The approach has little calibration costs, is low-cost, and offers low-latency per-formance comparable to more advanced camera-based or machine learning-based meth-ods and is therefore suitable for portable assistive devices.

Keywords: hand-gesture recognition; audio modulation; digital signal processing (DSP); assistive technology; human-computer interaction (HCI); Arduino Uno

 
 
Top