The needs of an individual with severe motor impairments may be significant, and they may be completely dependent on another person, in order to express their physiological and medical needs. Traditional assistive communication systems are expensive, space consuming, and require specialized interfaces to gain access, which are not accessible in low-resource environments.
This research aims to develop a cheap, real-time gaze-controlled communications system intended to improve assistive communication for users that have severe motor impairments. The system is implemented using Python and MediaPipe Face Mesh library to detect the landmarks of the eyes and perform gaze estimation for real-time processing. Facial landmarks are captured from the input stream using a webcam and a calibrated gaze-mapping algorithm is used to map the direction of eye gaze on a virtual on-screen keyboard.
The system comprises three crucial modules: (1) Eye Tracking Module (ETM) to extract facial landmarks with MediaPipe, (2) Gaze Mapping and Selection Module (GMSM) to detect fixations and select predefined keys based on the dwell-time, and (3) Communication Module (CM) to automatically send predefined messages to caregivers after fixing the key.
A dwell-time threshold mechanism, based on rules, is employed to reduce false activations, thereby achieving reliable selection of intended messages. It was tested in a controlled environment with the use of standard webcams and no special hardware. Experimental testing demonstrates consistent translation of gaze to command and responsive message triggering to meet real-time communication needs.
The presented framework shows that an affordable and software-based system of gaze interaction can greatly enhance the communication accessibility for people who are paralyzed or have severe motor impairments. The system is scalable to home-care and clinical applications because of its low computational and hardware requirements.