Hand Gesture to Sound: A Real-Time DSP-Based Audio Modulation System for Assistive Interaction

This paper presents the design, development, and evaluation of an embedded hardware and digital signal processing (DSP) based real-time gesture-controlled system. The system architecture utilizes an MPU6050 inertial measurement unit (IMU), Arduino Uno micro-controller, and Python-based audio interface to recognize and classify directional hand gestures, and transform them into auditory commands. Wrist tilts, i.e., left, right, forward, and backward, are recognized using a hybrid algorithm that uses thresholding, moving average filtering, and low-pass smoothing to remove sensor noise and transient errors. Hardware setup utilizes I2C-based sensor acquisition, onboard preprocessing on Arduino, and serial communication with a host computer running a Python script to trigger audio playing using the playsound library. Four gestures are programmed for basic needs: Hy-dration Request, Meal Support, Restroom Support, and Emergency Alarm. Experimental evaluation, conducted over more than 50 iterations per gesture in controlled laboratory setup, resulted in a mean recognition rate of 92%, with system latency of 120 to 150 milli-seconds. The approach has little calibration costs, is low-cost, and offers low-latency per-formance comparable to more advanced camera-based or machine learning-based meth-ods and is therefore suitable for portable assistive devices.